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INTRODUCTION 

The  principal  portion  of  this  Final  Report  on  the  present  grant.  Artificial 
Intelligence  Methods  in  Pursuit  Evasion  Differential  Games,  consists  of  on  at¬ 
tached  Technical  Report  by  Rodin  and  Weil,  on  Differential  Games  and  Artificial 
Intelligence  in  Air  Combat.  The  rest  of  the  report  is  a  brief  summary  of  our  activi¬ 
ties  and  achievements  under  the  grant  during  the  past  three  years.  The 
summary  is  brief,  because  it  is  merely  a  restatement  of  reports  sent  by  the  P.l.  to 
the  AFOSR  regularly  during  the  life  of  the  grant.  Since  those  reports  also 
contained  copies  of  publications  produced  under  the  sponsorship  of  this  grant, 
no  further  copies  of  such  publications  are  included  either.  However,  we  are 
including  copies  of  a  few  letters  and  other  material,  all  of  which  relate  to  items 
of  interest  mentioned  below. 


FULFILLMENT  OF  PROPOSED  WORK 

In  order  to  gauge  whether  we  succeeded  in  fulfilling  our  aims,  as  described  in 
the  Abstract  of  our  proposal  to  the  AFOSR  on  February  6, 1987,  we  quote  the 
Abstract: 


The  aim  of  the  research  here  proposed  is  to  develop  the 
conceptual  framework  and  the  software  for  the  prototype  of 
an  operational ,  on-board,  real  time  multiprocessing  computer 
system,  capable  of  assisting  the  pilot  in  flight  and  fire  control 
decisions;  in  other  words,  a  Tactical  Decision  Aiding  Expert 
System  ( TDAES ). 

The  end  product  of  this  research  will  be  for  use  in  theoretical 
combat  planning  and  analysis;  in  practical  fighter  pilot  train¬ 
ing  (e.g.,  in  simulations);  and  as  an  actual  aid  for  pilots  in  sup¬ 
port  of  their  tactical  decision  making  during  flights. 

The  nature  of  this  research  is  intelligent  control  of  a  very  spe¬ 
cific  type;  we  intend  to  combine  certain  aspects  of  differen¬ 
tial  game  theory,  3  dimensional  computational  geometry  and 
artificial  intelligence  in  a  unique  way,  so_as  to  provide  a  solu¬ 
tion  to  the  problem  described  above. ""  ( - 

We  believe  that  the  attached  Technical  Report  responds  in  meaningful  ways  to 
each  of  the  aims  set  forth  above.  Another  version  of  this  report  also  constitutes 
the  doctoral  dissertation  of  R.  Weil  under  the  direction  of  the  P.I.;  a  project  which 
was  initiated  shortly  before  the  starting  date  of  this  grant.  Therefore,  in  an  impor¬ 
tant  sense,  this  report  is  a  culmination  and  summing  up  of  much  of  the  work  we 
have  done  during  the  past  three  years.The  principal  subjects  of  the  Report  are 
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as  follows: 


1 .  Semantic  Control  Theory 

2.  Air  Combat  Environment 

3.  Pursuit-Evasion  Differential  Games 

4.  Artificial  Intelligence 

5.  Numerical  Algorithms 

6.  Multi-objective  Decision  Making 

7.  System  Integration 

8.  Simulation 

Thus,  to  be  specific,  of  our  aims  that  we  originally  set  forth,  the  only  one  missing  is 
that  of  multiprocessing.  That  is  a  subject  we  were  not  able  to  get  into  in  any 
meaningful  way,  because  of  lack  of  appropriate  computer  equipment. 


OTHER  PROPOSAL  FULFILLMENT  WORKS 

In  addition  to  the  Technical  Report  referred  to  above,  there  are  other  items  also, 
which  relate  to  the  fulfillment  of  our  purposes.  These  include  the  following: 

Doctoral  dissertations: 

1  .Artificial  Intelligence  Methods  in  Decision  and  Control  Systems;  by  Y.  Lirov;  un¬ 
der  the  supervision  of  E.Y.  Rodin. 

2.lntelliqent  Prediction  Methodologies  in  the  Navigation  of  Autonomous 
Vehicles:  by  M.  Amin;  under  the  supervision  of  E.Y.  Rodin. 

3 .Artificial  Intelligence  Methods  in  Utilizing  Low  Dimensional  Models  of  Differential 
Games:  by  R.D.  Weil;  under  the  supervision  of  E.Y.  Rodin. 


Other  publications: 

1  .'Artificial  Intelligence  in  Air  Combat  Games'  (E.Y.  Rodin,  L.W.  Wilbur,  B. 
McElhany,  S.  Mittnik  and  Y.  Lirov),  Pursuit-Evasion  Differential  Games,  ed.  by  Y. 
Yavin  and  M.  Pachter,  Pergamon  Press,  pps.  261-274, 1987. 

2. 'Annotated  Bibliography  of  Pursuit-Evasion  Differential  Games"  (E.Y.  Rodin), 
Pursuit  Evasion  Differential  Games,  ed.  by  Y.  Yavin  and  M.  Pachter,  Pergamon 
Press,  pps.  275-340, 1987. 

3.  'Artificial  Intelligence  Modelling  of  Control  Systems"  (E.Y.  Rodin,  Y.  Lirov,  B.G. 
McElhany  and  L.W.  Wilbur).  SIMULATION,  vol  50,  no.  1 ,  pps.  12-24, 1988. 
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4. “Semantic  Control  Theory"  (E.Y.  Rodin),  Appl.  Math.  Letters,  vol  1 ,  no.  1 ,  pps.  73- 
78, 1988. 

5. "Collision  Free  Path  Planning  in  a  Dynamic  Environment:  Semantic  Control 
Approach"  (E.Y.  Rodin,  B.K.  Ghosh,  F.  Golenko  and  R.W.  Weil),  SIMULATION,  vol 
50.  pps.  196-201,  1988. 

6.  "Automated  Learning  by  Tactical  Decision  Systems  in  Air  Combat"  (E.Y.  Rodin, 
B.J.  Ghosh  and  Y.  Lirov),  Comp,  and  Math,  with  Appl.,  vol  18,  no.  1-3,  pps.  151- 
160,  1989. 

7.  "A  Pursuit-Evasion  Bibliography  -  Version  2",  (E.Y.  Rodin),  Comp,  and  Math,  with 
Appl.,  vol  18,  no.  1-3,  pps.  245-320,  1989. 

8.  "Tactical  Air  Combat  Maneuvers:  Recognition  and  Guidance  Via  Neural 
Networks"  (E.Y.  Rodin  and  M.  Amin) ,  to  appear  in  Aerospace  America. 

9. "Flight  and  Fire  Control  with  Logic  Programming"  (E.Y.  Rodin  and  D.  Geist),  to 
appear  in  Comp,  and  Math,  with  Appl. 


Talks: 

In  addition  to  the  above  paper,  there  were  several  talks  given  by  the  P.l.  and/or 
his  graduate  students  at  various  meetings  and  conferences.  Examples  of  such 
venues: 

Aerospace  Applications  of  Artificial  Intelligence  Conferences 

International  Symposia  on  Differential  Games  Applications 

IEEE  Conferences  on  Decision  and  Control 

IEEE  International  Symposia  on  Intelligent  Control 

Society  for  Industrial  and  Applied  Mathematics  Conferences 

Naval  Underwater  Research  Center  Seminars 

Yale  University  Seminars 

Scott  Air  Force  Base  Seminars 


BENEFICIAL  EFFECTS  OF  THE  GRANT 

In  addition  to  the  results  enumerated  above,  there  were  several  additional  tan¬ 
gible  and  intangible  gains  made,  which  are  of  potential  utility  not  only  to 
Washington  University,  but  also  on  a  national  scale;  and  gains  which,  we  be¬ 
lieve,  are  of  potential  use  to  the  USAF  .  We  list  some  of  these  below. 
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Collaboration  with  Scott  AFB: 

An  excellent  working  relationship  developed,  in  the  course  of  this  grant,  be¬ 
tween  the  CINCMAC  Analysis  Group  on  the  one  hand,  and  the  P.1,  and  his 
graduate  students  on  the  other.  Items: 

1 .  A  group  of  30-40  students  of  the  P.l.  now  takes  a  regular  annual  field  trip  to 
Scott  AFB,  where  they  are  given  technical  presentations  by  various  AF  officers. 
Several  of  these  students  pursue  the  subjects  of  these  talks  in  minor  and  major 
research  projects. 

2.  We  arranged  two  special  seminar  series  this  past  school  year  at  Washington 
University,  relating  to  the  applications  of  Artificial  Intelligence,  specifically  for 
groups  from  Scott  AFB. 

3.  Scott  AFB  officer-scientists  gave  several  special  presentations  on  their  problem 
areas  to  groups  of  Washington  University  students,  at  the  University. 

4.  A  formal  agreement  of  internships  was  signed  between  Scott  AFB  and 
Washington  University. 

5.  Lt.  General  A.J.  Burshnick  nominated  the  P.l.  for  membership  to  the  Air  Force 
Scientific  Advisory  Board. 

6.  Scott  AFB  provided  funds  for  the  P.l.  to  equip  his  laboratory  with  a  SUN  4/260 
workstation. 

Other  Collaboration: 

The  P.I..  his  associates  and  graduate  and  undergraduate  students  gave  full  day 
progress  report  type  presentations  to  groups  from  each  of  the  following  organi¬ 
zation- 

AFOSR 

Scott  AFB  -  CINCMAC  Analysis  Group 
HQ  SAC  Delegation 
Rockwell  International 
McDonnell  Douglas 
Emerson  Electric 


RESEARCH  ENVIRONMENT 

As  a  direct  result  of  this  grant,  the  P.l.  was  enabled  to  establish  at  Washington 
University  the  following  two  entities: 

Semantic  Control  Laboratory 

This  laboratory  contains  all  of  the  computer  equipment,  which  was  utilized  for 
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the  purposes  of  this  grant,  and  which  will  continue  to  be  so  utilized.  It  also  con¬ 
tains  a  large  quantity  of  specialized  books  and  other  literature,  all  of  which  are 
specifically  earmarked  for  the  type  of  research  here  described.  The  laboratory 
turned  out  to  be  a  magnet  for  many  (specifically  American  undergraduate) 
students,  resulting  in  increased  interest  in  careers  related  to  science  and  engi¬ 
neering. 

Center  for  Optimization  and  Semantic  Control 

The  School  of  Engineering  at  Washington  University  agreed  to  establish  this 
Center  for  the  specific  purpose  of  both  fulfilling  the  aims  of  this  grant,  and  for  the 
purpose  of  enabling  the  P.l.  and  his  group  to  continue  their  research  of  the  na¬ 
ture  described  here.  E.Y.  Rodin  is  Director  of  the  Center;  its  members  are  drawn 
from  among  the  faculty,  and  from  among  graduate  and  undergraduate  stu¬ 
dents. 

In  addition  to  the  individual  and/or  joint  research  activities  of  members  of  the 
Center,  an  environment  is  created  which  is  conducive  for  drawing  the  interests 
of  some  of  our  brightest,  most  capable  and  most  motivated  students  towards 
our  interesting  research  projects. 

The  Center  holds  weekly  seminars,  at  which  ongoing  research  projects,  future 
efforts,  etc.  are  regularly  discussed. 


ACKNOWLEDGEMENT 
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DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  MILITARY  AIRLIFT  COMMAND 
SCOTT  AIR  FORCE  BASE.  ILLINOIS  6222S 


30  November  1989 


Dr  James  M.  McKelvey 

Dean,  School  of  Engineering  and  Applied  Science 
Washington  University 
St  Louis,  Missouri  63130 

Dear  Dr  McKelvey 

Just  a  short  note  to  thank  you  for  your  hospitality  and  the  superb  presenta¬ 
tions  of  Dr  Rodin,  his  colleagues  and  students.  We  were  most  impressed  with 
the  presentations— and  the  fact  that  so  many  students  were  willing  to  sacrifice 
their  Thanksgiving  holiday  to  meet  with  us.  They  are  doing  some  work  of  great 
interest  to  us  in  solving  real-world  problems.  We  hope  to  continue  this  rela¬ 
tionship  in  the  future. 

Thanks  again  for  taking  time  out  of  your  busy  schedule  to  meet  with  us. 


Deputy  Director,  Command  Analysis  Group 
DCS/Plans  and  Programs 


VOLUNTEER  SERVICE  AGREEMENT  BETWEEN  375  MSSQ/MSCS 
SCOTT  AFB,  IL  AND  WASHINGTON  UNIVERSITY,  ST.  LOUIS,  MO 

Statement  of  Understanding.  Volunteer  service  is  to  be  uncompensated  and  must  be  required  for 
academic  credit  or  certified  to  be  related  to  the  academic  process.  Participants  are  not  considered 
federal  employees  for  any  purpose  other  than  for  Federal  Tort  Claims  provisions  and  for  purposes 
relating  to  compensatory  injuries  sustained  during  the  performance  of  work  assignments.  This 
service  does  not  lead  to  noncompetitive  permanent  employment  with  this  base  or  with  any  other 
federal  installation. 

Though  the  university  will  refer  interested  students,  final  selection  is  left  to  the  Department  of  the 
Air  Force.  We  are  an  equal  employment  opportunity  employer  and  selection  will  be  based  on  these 
principles. 

The  undersigned  parties  understand  the  Volunteer  Service  Program  will  provide  meaningful 
experiences  reflecting  the  academic  requirements  of  the  participants  in  a  manner  so  as  not  to 
jeopardize  other  established  youth  program  positions  nor  positions  of  any  other  employee. 

Department  of  the  Air  Force  Volunteer  Service  Program  General  Objectives 

a.  To  provide  meaningful  assignments  which  relate  to  the  academic  endeavors  of  participants. 

b.  To  provide  experiences  which  enhance  the  career  education  and/or  career  development  of 
participants. 

c.  To  foster  amiable  relationships  between  representatives  of  the  Air  Force,  as  a  viable  potential 
employer,  and  university  officials. 

d.  To  encourage  women,  handicapped  individuals,  and  minorities  to  explore  career  options 

available  with  the  Department  of  the  Air  Force. 

e.  To  introduce  students  to  the  mission  and  functions  of  the  Air  Force  and  to  the  work 
environment  so  as  to  facilitate  their  development  of  an  appreciation  for  work  ethics. 

Activity  Responsibilities 

a.  Ensure  Office  of  Personnel  Management/Air  Force  Volunteer  Sendee  Program  objectives  are 
met. 

b.  Designate  a  Volunteer  Service  Program  Coordinator  to  whom  all  inquiries  can  be  directed. 

c.  Provide  structured  assignments  and  opportunities  for  experience  which  reflects  program 
objectives. 

d.  ?ertify  and  document  participation  in  and  completion  of  the  Volunteer  Service  Program. 

e.  Provide  all  participants  opportunities  to  evaluate  the  program  in  reference  to  program  objectives. 

f.  Ensure  that  this  in  no  way  jeopardizes  other  student  employment  programs  nor  contributes  to 
erosion  of  the  position  duties  of  other  employees. 
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JniYersity  ■Responsibilities 

i.  Develop  a  program  information  dissemination  system  which  ensures  that  all  students  are  given 
equal  opportunity  to  apply  for  referral. 

5.  Provide  activity  with  name  of  school  officials  to  whom  service  inquiries  can  be  directed. 

:.  Provide  students'  resumes  to  activity. 

i.  Provide  information  necessary  for  activity  to  provide  structured,  meaningful  assignments  when 
academic  assignments  require  such. 

s.  Inform  activity  of  any  change  in  participants'  status. 

Participant  Responsibilities 

a.  Though  Volunteer  Program  participants  are  not  federal  employees,  they  are  expected  to  maintain 
acceptable  attendance  and  to  conduct  themselves  so  as  not  to  disrupt  normal  workflow  in  the 
activity. 

b.  Provide  information  which  may  be  needed  for  certain  reporting  requirements. 


Civilian  Personnel  Officer 


14/  .  tyMCi 


fan,  School  of  Engineering 
shington  University,  St.  Loui 


,  MO 


3-S-lo 

Date 
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THE  WASHINGTON  UNIVERSITY/MAC  INTERN  PROGRAM 


The  Intern  Program  with  Washington  University  teams  an  undergraduate  student 
with  analysts  from  the  Command  Analysis  Group  on  an  issue  as  it  is  being  analyzed 
for  the  senior  executives  at  the  Military  Airlift  Command.  This  teaming  provides  a 
symbiotic  relationship.  The  analysts  benefit  from  learning  from  the  student  some  of 
the  newer  innovative  methodologies  being  taught  at  Washington  University.  The 
student  experiences  the  cycle  of  analysis  from  the  initial  nebulous  statement  of  the 
issue  through  the  presentation  of  the  analytical  findings  to  senior  executives  for  a 
decision.  We  want  to  motivate  students  to  continue  a  technical  course  of  study  by 
giving  them  a  "piece  of  the  action"  and  letting  them  see  what  practitioners  do.  We 
are  trying  to  help  alleviate  the  national  problem  of  low  graduate  enrollment  in  the 
sciences  and  engineering  by  American  students. 


DEPARTMENT  OF  THE  AIR  FORCE 

headquarters  strategic  air  command 

OFFUTT  AIR  FORCE  BASE,  NEBRASKA  68113 


Ervin  Y.  Rodin 

Box  1040,  Washington  University  1  g  ^ggg 

One  Brookings  Drive 

St.  Louis,  MO  63130-4899 

Dear  Dr  Rodin 

1.  As  per  your  request  from  our  recent  telephone  conversation,  listed  below 
are  the  individuals  planning  to  visit  your  facilities. 

Dr.  Jacqueline  Henningsen,  Chief,  Capability  Assessment  Division, 

Strategic  Air  Command 

Mr.  Patrick  McKenna,  Operations  Research  Analyst, 

Strategic  Air  Command 

Dr.  Fred  Choobineh,  Dept  of  Industrial  Engineering, 

University  of  Nebraska,  Lincoln 

2.  We  can  arrive  at  your  facilities  by  10:00  AM  on  Monday,  April  2,  1990.  I 
line  with  your  suggestion,  please  send  introductory  materials  describing  your 
program  and  a  possible  agenda  for  the  day.  We  look  forward  to  learning  about 
your  progress. 

Sincerely 

l^r^iacqu^ine  R.  Henningsen,  GM-15 

Chief,  Capability  Assessments  Div 

Dir  Force  Assessments,  DCS/Plans  &  Resources 


Fax  #  (402)  294-7628 
Voice  #  (402)  294-2355 


Washington 


WASHINGTON  •  UNIVERSITY’  IN’  ST-  LOUIS 


School  of  linginccrinj*  and  Applied  Science 


Center  for  Opiimi/uition  and  Semantic  Control 


To:  Rll  ssm  (and  other  interested)  undergraduate  and 
graduate  students 

From:  Professor  6.  V.  Rodin 


On  Friday,  march  2,  1990,  a  group  of  scientists  from 
Scott  flir  Force  Base  uuill  give  a  l-hour  talk  in  the  Berger 
Room  (Cupp.  II,  101)  at  12:00  noon.  They  uuill  present  a 
set  of  as  yet  unsolved  optimization  problems,  uuhich 
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and 
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A  new  methodology  is  developed  for  dealing  with  complex  systems  that 
are  self  modifying  in  the  order,  relations,  and  component  types.  Air-combat  is 
one  such  system.  Differential  game  technology  currently  and  for  the  foreseeable 
future  is  incapable  of  generating  solutions  to  all  but  very  low  dimensional  models 
for  air-combat.  The  solutions  of  the  low  dimensional  models  are  by  themselves 
too  simplistic  to  be  useful  in  actual  air-combat. 

Artificial  intelligence  methodologies  and  the  new  Semantic  Control 
Paradigm  splice  together  suboptimally  the  low  dimensional  differential  game 
solutions.  Mesarovic’s  general  systems  theory  is  extended  by  defining  for  the 
first  time  Semantic  Systems  dealing  with  systems  that  self  modify  their  form 
(order  and  relations).  A  new  methodology  for  simulation  of  Semantic  Systems 
based  on  Al  Frames  is  also  developed. 

Three  artificial  intelligence  methodologies  are  explored  for  splicing  the 
differential  games:  rule  based,  the  Analytical  Hierarchy  Process,  and  artificial 
neural  nets.  The  rule  based  approach  uses  an  explicit  set  of  rules  in  a  forward  or 
backward  chaining  fashion  to  determine  the  most  appropriate  differential  game. 
The  artificial  neural  net  approach  is  based  on  a  paradigm  modelled  after 
biological  processes.  Simulation  trains  a  hierarchy  of  neural  nets  to  determine 
the  most  applicable  differential  game.  The  Analytical  Hierarchy  Process  is  a 
paradigm  of  relative  measure.  First  determined  is  the  relative  importance  of  a  set 
of  criteria  for  choosing  the  most  appropriate  differential  game.  Next  determined 
with  respect  to  each  criterion  is  a  relative  rating  of  each  differential  game 
alternative.  A  tallied  score  determines  the  game. 

Those  aspects  of  differential  game  theory  that  are  useful  for  air-combat 
are  reviewed.  Tne  game  of  kind  determines  the  region  of  state  space  from  which 


capture  of  an  adversary  is  possible.  The  game  of  degree  determines  trajectories 
based  on  a  cost  function  in  the  region  of  state  space  in  which  capture  is  possible. 
A  simplified  development  of  the  necessary  conditions  for  the  game  of  degree  is 
given  by  combining  Berkovitz’s  methodology  with  Isaac's  theorem  (that  all  games 
are  equivalent  to  an  autonomous  game  with  terminal  payoff).  Berkovitz’s  result, 
depended  always  on  using  the  family  of  time  varying  feedback  strategies  in  the 
variation,  has  been  extended  to  consider  autonomous  feedback  strategies  for 
autonomous  systems. 

The  overall  approach  taken  in  this  work  has  been  to  combine  different 
technologies  that  by  themselves  are  currently  incapable  of  solving  a  problem. 
Hybridization  techniques  for  future  integrated  system  methodologies  may  solve 
problems  that  are  currently  beyond  the  scope  of  pure  solutions.  Methodologies 
in  this  work  have  application  to  problems  deemed  so  difficult  that  no  appropriate 
mathematical  models  can  be  derived,  nor  can  be  found  any  appropriate  analytical 
methods. 
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A  new  methodology  is  developed  for  dealing  with  complex  systems  that 
are  self  modifying  in  the  order,  relations,  and  component  types.  Air-combat  is 
one  such  system.  Differential  game  technology  currently  and  for  the  foreseeable 
future  is  incapable  of  generating  solutions  to  all  but  very  low  dimensional  models 
for  air-combat.  The  solutions  of  the  low  dimensional  models  are  by  themselves 
too  simplistic  to  be  useful  in  actual  air-combat. 

Artificial  intelligence  methodologies  and  the  new  Semantic  Control 
Paradigm  splice  together  suboptimally  the  low  dimensional  differential  game 
solutions.  Mesarovic's  general  systems  theory  is  extended  by  defining  for  the 
first  time  Semantic  Systems  dealing  with  systems  that  self  modify  their  form 
(order  and  relations).  A  new  methodology  for  simulation  of  Semantic  Systems 
based  on  Al  Frames  is  also  developed. 

Three  artificial  intelligence  methodologies  are  explored  for  splicing  the 
differential  games:  rule  based,  the  Analytical  Hierarchy  Process,  and  artificial 
neural  nets.  The  rule  based  approach  uses  an  explicit  set  of  rules  in  a  forward  or 


backward  chaining  fashion  to  determine  the  most  appropriate  differential  game. 
The  artificial  neural  net  approach  is  based  on  a  paradigm  modelled  after 
biological  processes.  Simulation  trains  a  hierarchy  of  neural  nets  to  determine 
the  most  applicable  differential  game.  The  Analytical  Hierarchy  Process  is  a 
paradigm  of  relative  measure.  First  determined  is  tne  relative  importance  of  a  set 
of  criteria  for  choosing  the  most  appropriate  differential  game.  Next  determined 
with  respect  to  each  criterion  is  a  relative  rating  of  each  differential  game 
alternative.  A  tallied  score  determines  the  game. 

Those  aspects  of  differential  game  theory  that  are  useful  for  air-combat 
are  reviewed.  The  game  of  kind  determines  the  region  of  state  space  from  which 
capture  of  an  adversary  is  possible.  The  game  of  degree  determines  trajectories 
based  on  a  cost  function  in  the  region  of  state  space  in  which  capture  is  possible. 
A  simplified  development  of  the  necessary  conditions  for  the  game  of  degree  is 
given  by  combining  Berkovitz's  methodology  with  Isaac’s  theorem  (that  all  games 
are  equivalent  to  an  autonomous  game  with  terminal  payoff).  Berkovitz’s  result, 
depended  always  on  using  the  family  of  time  varying  feedback  strategies  in  the 
variation,  has  been  extended  to  consider  autonomous  feedback  strategies  for 
autonomous  systems. 

The  overall  approach  taken  in  this  work  has  been  to  combine  different 
technologies  that  by  themselves  are  currently  incapable  of  solving  a  problem. 
Hybridization  techniques  for  future  integrated  system  methodologies  may  solve 
problems  that  are  currently  beyond  the  scope  of  pure  solutions.  Methodologies 
in  this  work  have  application  to  problems  deemed  so  difficult  that  no  appropriate 
mathematical  models  can  be  derived,  nor  can  be  found  any  appropriate  analytical 
methods. 
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ARTIFICIAL  INTELLIGENCE  METHODS  IN  UTILIZING  LOW  DIMENSIONAL 
MODELS  OF  DIFFERENTIAL  GAMES 


1.  INTRODUCTION 

One  of  the  most,  if  not  the  most  complex  dynamical  systems  is  the  air- 
combat  arena.  The  demand  on  a  pilot’s  faculties  becomes  more  demanding  at 
each  incremental  increase  in  the  level  of  technology  at  both  the  pilot's  and  his 
adversaries'  disposal.  Thus,  the  purpose  of  this  research  is  the  development  of  a 
real-time  tactical  flight  and  fire  control  system.  The  development  of  this  system 
will  consider  the  capabilities  of  the  aircraft,  weapons,  and  electronic 
countermeasures  of  both  the  pilot  and  his  adversaries. 

Rufus  Isaacs  first  looked  into  the  theory  of  modeling  tactical  encounters  in 
what  he  termed  "Differential  Games"  in  his  seminal  Rand  report.  Isaacs 
assumed  a  differential  model  for  the  aircraft  dynamics.  He  also  assumed  fixed, 
the  roles  of  pursuer  and  evader  for  the  duration  of  an  encounter.  Only  the 
pursuer  had  weapon  capabilities.  Hyper-surfaces  in  the  state  space  modeled 
weapon  capabilities.  The  weapon  model  supposed  that  if  the  state  of  the  system 
was  driven  to  the  aforementioned  hyper-surface  then  the  evader  is  destroyed 
with  probability  one,  while  off  the  surface  there  is  probability  zero  that  the  evader 
is  destroyed. 
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There  are  many  reasons  differential  games  have  not  found  widespread 
use  in  air  combat.  Isaacs'  conditions  are  only  "necessary"  conditions.  The 
meeting  of  these  conditions  does  not  guarantee  a  solution,  but  a  solution  must 
meet  these  conditions.  It  has  only  been  possible  to  find  solutions  to  very  low 
order  models  for  an  encounter  (e.g.,  two  dimensional  constant  velocity).  This 
happens  because  due  to  the  complex  geometric  structure  of  the  solutions,  the 
controls  may  be  noncontinuous  across  manifolds  in  the  state  space. 

The  intractability  of  finding  solutions  to  sufficiently  high  order  models  for 
fixed  role  encounters  is  not  the  only  impediment  to  the  use  of  differential  games. 
The  assumption  that  weapon  systems  can  be  modeled  by  a  surface  in  the  state 
space  such  that  off  the  surface  there  is  zero  probability  of  destruction  of  an 
adversary,  and  on  the  surface  there  is  probability  one  of  destruction  is  not 
realistic.  A  continuous  probabilistic  model  that  assigns  to  every  point  in  state 
space  a  probability  of  destroying  an  adversary  is  a  more  realistic  weapon 
systems  model.  This  probability  function  depends  on  such  items  as  range,  range 
rate,  aspect  angle,  off  boresight  angle,  etc.  Clearly,  it  is  easy  to  include  the 
effects  of  electronic  countermeasures  in  this  function  by  adjusting  the 
probabilities  in  an  appropriate  fashion.  The  fixed  role  assumption  during  an 
encounter  is  also  not  reasonable.  All  participants  in  an  encounter  have  weapon 
systems  that  can  be  modeled  by  continuous  probabilistic  functions.  Each 
participant  tries  to  obtain  a  reasonable  firing  opportunity  while  preventing  his 
adversaries  from  doing  the  same. 

The  solution  to  the  low  order  one  target  models  are  by  themselves,  not 
sufficiently  accurate  to  be  useful  in  air  combat.  It  is  the  contention  of  my 
research  that  artificial  intelligence  methodologies  may  be  used  to  splice  these  low 
order  solutions  together  in  a  sub-optimal  fashion  that  will  be  useful  in  air  combat. 
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The  first  step  in  solving  the  air  combat  problem  is  defining  how  to  model 
such  a  complex  system.  The  complexity  of  the  air  combat  arena  falls  outside 
current  definitions  of  general  systems  as  first  proposed  by  Mesarovic.  The  basis 
for  traditional  system  definitions  are  a  fixed  set  of  objects  and  a  fixed  set  of 
relations  between  the  objects.  Our  interests  are  systems  that  have  the  property 
that  they  self  modify.  This  modification  takes  place  by  attrition  and  reproduction 
of  the  objects,  and  modification  of  relations  between  the  objects.  The  objects  of 
course  are  aircraft,  missiles,  SAMs,  and  ground  targets. 

One  area  of  my  research  explores  the  characteristics  of  self  modifying 
systems  of  the  above  type.  I  term  these  type  of  systems,  semantic  systems. 
Semantic  systems  are  shown  to  be  an  extension  of  general  systems  defined  by 
Mesarovic.  The  semantic  state  object  consists  of  terms  representing  the  order  of 
the  system,  and  the  relations  among  them. 

Closed  form  analytical  techniques  are  generally  only  useful  for  very  low 
order  system  models.  Simulation  techniques  must  be  used  for  understanding 
system  models  of  any  real  complexity  for  even  classical  general  systems.  The 
normal  case  assumes  that  a  system  may  be  classified  according  to  such 
categories  as,  continuous  dynamical  systems,  discrete  dynamical  systems, 
linguistic  systems,  decision  systems,  etc.  Closed  form  analytical  tools  assume 
that  a  system  (or  its  model)  is  of  a  homogeneous  classification.  The  problem  that 
we  have  to  deal  with  is  that  even  the  portions  of  the  air-combat  arena  that  can  be 
classified  as  a  Mesarovic  general  system  are  not  of  a  homogeneous  nature.  The 
fact  that  the  system  is  self  modifying  further  compounds  the  problem.  I  also  will 
introduce  a  new  technique  for  modeling  systems  of  the  aforementioned  complex 
nature.  The  basis  of  this  technique  is  Marvin  Minsky’s  artificial  intelligence 
frames.  I  show  that  the  frame  based  modeling  and  simulation  paradigm  is 
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natural  for  semantic  systems.  This  allows  one  to  define  the  systems  objects  and 
relations  (semantic  state)  in  terms  of  declarative  and  procedural  knowledge. 

The  semantic  control  paradigm  will  be  used  to  determine  the  controls  for 
an  aircraft.  The  control  problem  breaks  into  three  blocks. 

(i)  Identifier 

The  identifier  fuses  sensor  data  to  estimate  the  current  semantic 
state.  Component  types  (MIGS,  SAMS,  Airliners)  and  state  are 
estimated.  Also  estimated  are  the  relations  between  the  identified 
components  (attacking,  retreating).  Sensor  fusion  is  itself  a 
complex  subject  of  current  research,  but  is  beyond  our  interest 
here.  We  assume  complete  information  of  the  semantic  state. 

(ii)  Goal  Selector 

The  Goal  Selector  determines  the  most  appropriate  differential 
game  based  on  the  identified  semantic  state.  The  choice  is  made 
from  a  knowledge  base  of  games  from  which  solutions  can  be 
generated.  The  results  are  the  optimal  trajectories,  barrier,  and 
controls.  It  should  be  noted  that  these  controls  are  not  the  same 
controls  as  those  needed  by  the  aircraft,  because  the  game  chosen 
may  be  a  simplification  of  the  actual  encounter.  This  block  has  a 
library  of  numerical  methods  used  to  solve  the  differential  games. 

(iii)  Adapter 

The  Adapter  determines  the  controls  that  cause  the  aircraft  to 
"best"  follow  the  optimal  trajectory  determined  by  the  Goal  Selector. 
This  is  best  accomplished  by  a  rule-based  control  system 
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dependent  upon  the  differential  game  model  chosen  in  the 
identifier,  and  upon  the  performance  limits  of  the  aircraft. 

The  design  of  the  goal  selector  is  the  main  thrust  of  this  dissertation. 
Three  alternate  methodologies  are  considered:  rule  based,  the  Analytical 
Hierarchy  Process,  and  artificial  neural  nets.  The  rule  based  approach  uses  an 
explicit  set  of  rules  in  a  forward  or  backward  chaining  fashion  to  find  the  most 
appropriate  differential  game.  The  artificial  neural  net  approach  is  based  on  a 
paradigm  modelled  after  biological  processes.  Simulation  trains  a  hierarchy  of 
neural  nets  to  find  the  most  applicable  differential  game.  The  Analytical 
Hierarchy  Process  is  a  paradigm  of  relative  measure.  First  determined  is  the 
relative  importance  of  a  set  of  criteria  for  choosing  the  most  appropriate 
differential  game.  Next  determined  with  respect  to  each  criterion  is  a  relative 
rating  of  each  differential  game  alternative.  A  tallied  score  determines  the  game. 

The  use  of  Pursuit-Evasion  games  gives  us  the  ability  to  execute  actions 
in  at  least  locally  optimal  trajectories,  which  is  something  that  a  human  pilot  does 
not  automatically  achieve.  However  most  Pursuit-Evasion  solutions  are  over 
simplifications  of  real  life  air  combat.  These  solutions  need  to  be  refined,  and  this 
has  to  be  done  when  experience  is  acquired  by  the  utilization  of  this  system. 
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2.  DIFFERENTIAL  GAMES 

This  first  section  covers  the  aspects  of  differential  game  theory  applicable 
to  air-combat.  Contained  are  the  derivations  of  the  necessary  conditions  for  the 
game  of  kind  and  game  of  degree.  The  derivation  of  the  necessary  conditions  for 
the  game  of  degree  extends  Berkovitz's  work  to  consider  only  autonomous 
variations  for  autonomous  systems.  Section  6  uses  the  results  derived  in  this 
section  in  generating  the  solutions  for  the  differential  games  in  the  knowledge 
base. 


Rufus  Issacs  in  his  seminal  Rand  Reports  first  introduced  the  notion  of 
what  he  called  differential  games.  He  formulated  the  idea  of  two  opposing  players 
having  control  of  a  system  described  by  differential  equations.  One  of  the 
players  (pursuer)  wishes  to  drive  the  state  of  the  system  to  penetration  of  a 
terminal  surface  (manifold)  while  the  other  player  (evader)  wishes  to  prevent 
penetration.  Isaacs  defined  two  complementary  methodologies  of  solution  to  this 
problem,  games  of  degree  and  games  of  kind. 

The  game  of  kind  determines  from  what  regions  of  the  state  space 
penetration  of  the  terminal  surface  is  possible.  The  game  of  kind  determines 
boundaries  that  separate  the  state  space  into  regions  of  capture  and  escape.  The 
capture  region  is  defined  as  the  set  of  all  points  from  which  the  pursuer  can  force 
penetration  of  the  terminal  surface  regardless  of  the  controls  used  by  the  evader. 
Similarly,  the  escape  region  is  defined  as  the  set  of  all  points  from  which  the 
evader  can  prevent  penetration  of  the  terminal  surface  regardless  of  the  controls 
of  the  pursuer.  The  boundaries  that  separate  the  capture  and  escape  region  are 
generated  by  barriers  that  neither  the  pursuer  nor  evader  can  force  to  be 
penetrated.  If  the  barriers  do  not  form  the  boundary  of  a  closed  region  of  the 
game  space  then  capture  is  possible  from  the  entire  game  space.  The  barriers 
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then  form  surfaces  that  force  classical  swerve  manuevers  in  the  trajectories  of 
the  players. 

The  game  of  degree  determines  what  the  optimal  controls  and  trajectories 
of  the  players  should  be  in  the  capture  region.  This  is  done  by  defining  a  cost 
function  on  the  controls,  state  space,  and  terminal  surface.  The  solution  to  the 
game  of  degree  may  not  be  unique,  and  exists  only  for  those  initial  points  from 
which  it  is  possible  for  the  pursuer  to  drive  the  state  to  the  terminal  surface 
regardless  of  the  controls  of  the  evader,  i.e.  for  points  contained  in  the  capture 
region  determined  by  the  game  of  kind. 
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2.1.  GAMES  OF  DEGREE 

The  necessary  conditions  for  the  game  of  degree  are  now  derived.  Our 
methodology  follows  Berkovitz  (1967)  using  a  theorem  of  Isaacs  (Theorem  2.1.1 
page  10)  for  a  simplified  presentation.  Lemma  2.1.3  and  Theorem  2.1.2  are 
original  contributions  and  extend  Berkovitz’s  work  to  consider  only  autonomous 
variations  for  autonomous  systems.  Berkovitz  always  found  it  necessary  to  use 
the  class  of  time  varying  variations. 

2.1.1.  GAME  SPACE 

Let  X  be  an  n-dimensional  vector,  Y  be  an  s-dimensional  vector,  and  Z  be 
an  sA-dimensional  vector.  The  game  space  E  is  a  subset  of  n-dimensional 
Euclidean  space  bounded  by  the  terminal  manifold  C.  Then  X  e  E  describes  the 
position  in  the  game  space  and  can  be  described  by  differential  equations: 

(2.1.1)  Xj#  =  fj(X,Y,Z);  j  =  1,2, ...,n 

The  vector  valued  function  f(X,Y,Z)  =  (f-j  (X,Y,Z),...  ,fn(X,Y,Z))  is  assumed  to  be  of 
class  C^1 )  on  it  arguments.  The  vectors  Y  and  Z  are  the  control  vectors  of  the 
pursuer  and  evader  respectively. 

2.1.2.  TERMINAL  MANIFOLD 

The  game  ends  when  the  trajectory  penetrates  an  (n-l)-dimensional 
terminal  manifold  C.  The  manifold  C  is  at  least  piecewise  continuous  and  on 
each  section  is  C ^ ).  We  assume  we  can  parameterize  C  as: 

(2.1.2)  X  =  h(s-,,s2 . sn_f) 


where  h  in  a  n-dimensional  vector  function. 
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2.1.3.  PAYOFF 


The  game  of  degree  has  integral  payoff  of  the  form: 


(2.1.3) 


P(Xo,«M»)  = 


5f 


J  G(X,<t>,<D)dt  +  H(s) 
x0 


where 

H  s  A  smooth  function  defined  on  (n-1)  dimensional  terminal  manifold  C 
G  s  A  function  containing  partials  of  any  order 

4>  s  The  control  strategy  of  the  pursuer,  a  vector  valued  function  <>:  X  — >  V 
O  =  The  control  strategy  of  the  evader,  a  vector  valued  function  <I>:  X  -» Z 
tf  s  The  time  the  trajectory  first  contacts  the  terminal  manifold  C. 
t0  =  Initial  time  of  game. 

X0  a  X(t0) 


A  game  of  degree  is  said  to  have  integral  payoff  if  H  ■  0,  and  terminal  pavoff  if  G 
=  0.  A  game  with  integral  payoff  is  said  to  be  a  pursuit  evasion  game  if  G  =  1 ,  and 
a  game  of  survival  if  G  is  not  constant. 

The  following  theorem  allows  us  to  derive  theoretical  results  strictly  in 
terms  of  games  of  degree  with  terminal  payoff. 
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THEOREM  2.1.1  i1  A  game  with  payoff  G  *  0  can  be  replaced  by  an  equivalent 
one  with  terminal  payoff. 

This  is  done  by  adjoining  the  new  state  variable  Xn+-J ,  new  terminal  cost  H\  new 
terminal  manifold  C\  and  new  game  space  E  such  that: 

X*n+1  =  G(X,<j>,<D) 

H’(s')  =  H(s)  +  sn 

Xn .  i  (tf)  =  sn  (terminal  condition) 

(2.1.4) 

Xn+i(0)  =  0  (initial  condition) 

E'  =  E  x  (-oo.oo) 

C'  =  C  X  (-«>,<») 


This  theorem  can  be  extended  so  that  our  following  results  are  valid  for 
other  forms  of  games  not  directly  incorporated  so  far.  If  t  effectively  appears  in  fj, 
G.  or  H  adjoin  X*n+i  =  1,  replace  t  by  Xn+1,  use  Xn+1(0)  =  0,  F*  Ex  (-«,»), 
and  C’=  C  x  (-oo.oo).  Games  of  fixed  duration  can  also  be  incorporated,  e.g. 


(2.1.5) 


P(Xo^O)  = 


*T 

G(X,<j),0)dt 

0 


This  is  done  by  adjoining  T*  =  Xn+i  =  -1,  Xn+i  (0)  =  T,  C'  =  Xn+1  =  0,  and  E’=  E 

X  (-oo.oo) 


The  goal  of  the  pursuer  will  be  to  minimize  the  payoff,  while  the  goal  of  the 
evader  will  be  to  maximize  the  payoff. 


1 1saacs  Sec.  2.4,  p.g.  1 
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2.1.4.  NECESSARY  CONDITIONS 

We  will  now  derive  the  necessary  conditions  for  the  optimal  strategies  of 
both  players.  These  derivations  will  follow  Berkovitz  and  we  will  assume  the 
existence  of  optimal  strategies  (feedback  control  laws).  This  differs  from  Isaacs’ 
heuristic  treatment  in  which  he  only  assumed  the  existence  of  a  single  optimal 
trajectory.  Isaacs’  theorem  that  any  non-autonomous  game  with  integral  cost  can 
be  replaced  by  an  equivalent  autonoumous  game  with  terminal  cost  will  also  be 
used  to  simplify  the  derivations.  Berkovitz  used  non-autonomous  games 
because  his  variations  of  strategies  required  variations  in  optimality  over 
arbitrarily  small  increments  of  time  even  for  autonomous  games2.  We  use  the 
notion  of  level  sets  to  consider  only  autonomous  strategies  for  autonomous 
games. 

The  following  assumptions  will  be  needed.  Without  loss  of  generality  we 
have  shown  (Theorem  2.1.1)  that  only  terminal  payoff  functions  need  be 
considered.  The  payoff  function  is  simply  P(Xq,<)>,<I>)  =  H(Xf).  The  closure  of  E 
(Ep)  is  contained  in  a  bounded  subset  n  of  (X)-space.  We  also  assume  the 
existence  of  the  bounded  subset  r  of  (X,4»,<l>)-space  which  contains  the  projection 

of  n.  C  may  be  decomposed  into  Cj  ,i  =  1,  2 . a,  (n-l)-dimensional  connected 

sub-manifolds  of  class  each  contained  in  n  such  that: 

a 

(2.1.6)  C=UCj 

H  1 


2Berkovitz  considered  the  family  of  non-autonomous  strategies  even  for 
autonomous  games. 
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The  (n-l)-dimensional  sub-manifolds  are  each  parameterized  as: 

(2.1.7)  X  =  hj(s1(s2 . sn_-|)  j  *  1,2 . a 

2.1.5.  STRATEGIES 

The  underlying  assumption  is  that  optimal  strategies  exist  for  both  the 
pursuer  and  evader.  Let  U  denote  the  class  of  functions  <f>  that  are  piecewise 
CO)  mappings  of  X  on  E°  to  s-dimensional  Euclidean  space.  Let  V  denote  the 
class  of  functions  O  that  are  piecewise  CO)  mappings  of  X  on  E°  to  sA- 
dimensional  Euclidean  space. 

Let  <}>  e  U  and  C>  e  V,  then  the  differential  equations  describing  the  motion 
in  the  state  space  is: 

(2.1 .8)  X*  =  f(X,<J>(X),<D(X));  X(t0)  =  X0 

There  may  be  more  than  one  solution  to  this  differential  equation  if  Xg  is  point  of 
discontinuity  of  <j>  or  <t>.  A  solution  that  is  unique  in  a  neighborhood  of  Xq  may 
bifurcate  at  some  future  point  of  the  trajectory  at  a  point  of  discontinuity  of  <{>  or  O. 
Isaacs  termed  these  surfaces  of  discontinuity  of  <j>  or  <I>  semipermeable 
surfaces.  Some  of  these  surfaces  to  be  explored  later  are  barriers,  dispersal,  and 
universal  surfaces. 

Berkovitz  defines  a  playable  pair  as  p  e  U  and  O  e  V  such  that  for  every 
Xq  e  E,  X  reaches  C  in  finite  time.^  This  puts  the  restrictions  on  the  pursuer,  as 
the  goal  of  the  evader  is  to  avoid  penetration  of  C  as  long  as  possible.  The 
underlying  assumption  is  that  E  is  the  capture  region  so  there  must  exist  at  least 
one  <t>  for  every  <D  that  satisfies  the  playability  condition.  The  payoff  function 
P(Xq,<J>,<I>)  =  H(Xf),  exists  for  each  playable  pair  (0,  <D)  and  Xq  e  E  but  may  be 


^Berkovitz  p.g  4 
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multivalued.  The  capture  region  assumption  guarantees  the  non-empty  maximal 
pair  of  subclasses  ^dj,  c  V  having  the  property  that  every  pair  $  e  U M, 
<D  e  is  playable.  The  members  of  and  are  called  pure  strategies. 

The  roles  in  the  game  are  that  the  pursuer  should  choose  a  strategy  <}>  € 
L/m  to  minimize  P(Xq,0,<D).  The  evader  should  choose  a  strategy  <t>  e  to 
maximize  P(XoA<&)-  The  playable  pair  such  that  P(Xo,<j>*,<J>*)  is  single 

valued  is  a  saddle  point  if 

(2.1 .9)  P(X0,<j>>)  <  P(X0V ,<D*)  <  P(X0 A**) 

holds  for  every  Xg  e  E,  < j>  e  Uy\,  and  O  e  V^.  The  game's  optimal  strategies  are 
then  and  <d\  The  value  function  of  the  game  is  then  defined  as  W(Xg)  = 
P(Xg,(j>*,0*).  The  optimal  trajectory  corresponding  to  (<>* ,0*)  is  denoted  o*(Xg,t). 

2.  •  .6.  GAME  SPACE  DECOMPOSITION 

We  will  now  look  at  decomposing  the  game  space  into  regions  of 
continuity  and  discontinuity  of  the  optimal  controls  <>*  and  <D*.  The  discontinuities 
are  the  semi-permeable  surfaces  mentioned  earlier.  The  semi-permeable 
surfaces  are  sub-manifolds  of  dimension  (n-1)  and  serve  to  separate  the  game 
space  E  into  regions  of  continuity. 

A  regular  decomposition  of  the  game  space  E  is  a  decomposition  into 

subregions  denoted:  E^,  E^2<  £-|ji,  ^21 .  ^2j2>  — >^a1 .  ^aja 

such  that  the  E,j  satisfy  the  following  properties:4 


4Remember  a  is  the  number  of  components  in  the  decomposition  of  the  terminal 


manifold  C. 
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(a.)  Ey,  i  =  j  =  1,2,. ...jj,  is  connected  and  has  piecewise 

smooth  boundary. 

(2.1.10)  (b.)  EjjnEk|  =  0i*korj*l. 

.  (c.)  £°  =  U  E0.5 

i.j 

The  regions  Ej,  i=1,2,...,a,  defined  as 


(2.1.11)  E  =  di  (£C„  n£] 


are  connected,  have  piecewise  smooth  boundary,  and 

(2.1.12)  £joEj  =  0i^j. 


The  regions  Ej  i=  1,2,..., a  are  such  that: 

(a.)  The  E  always  lie  on  the  same  side  of  the  C; 

(2.1.13) 

(b.)  £°j  n  Ck  =  0  for  i  ^  k. 


We  also  note  that  for  each  i  =  1 ,2,...,a: 


(2.1.14) 


(a.)  E^jj  n  Cj  *  0 
(b.)  E0^  n  Cj  =  0  for  j  ^  jj. 


The  sub-manifolds  My,  i  =  1 ,2 . a,  and  j  =  1 ,2 . j,-1  defined  by, 

(2.1.15)  /Wlj  =  (Ecijn£Ciij+1)nE, 

are  oriented,  connected,  of  dimension  (n-1),  and  of  class  C^).  We  assume  the 
sub-manifolds  can  be  parameterized  by  si  =  (si-j  ,si2.---.s^n-i )  such  that, 

(2.1.16)  X  =  hij(s)1,si2 . sip^). 

The  sub-manifolds  My  then  divide  Ej  into  two  disjoint  regions.  The  Afy  i  = 
1 ,2,...,a,  and  j.k  =  1 ,2,...,jj-1  also  satisfy: 

(2.1.17)  McynMcik  =  0forj*k. 


5Recall  that  E^y  is  the  closure  of  the  set  Ey 
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The  sub-manifolds  N\ i  \2  ik  defined  for  each  subset  i-|  ,i2**--»‘K  ,n^e9®rs 
i=1,2 . a  as 

(2.1.18)  /Vj-j  |2 . jk  =  (^ji  n  &*'\2  n  •••  n  f°ik)  °  E  are  ©fther  empty  or  a 

connected  non-singular  oriented  differentiable  manifold. 


EXAMPLE  REGULAR  DECOMPOSITION 


Some  final  notation  is  needed  before  we  close  out  this  section.  The 
sections  Cj  of  the  terminal  manifold  will  often  be  denoted  as  Mj  jj.  /Wj0  will  denote 
the  union  of  all  the  Nji  j2 ,...,ik  such  that  ■  e  *1  »'k*  Last|y  ^or 

i  =  1 ,2 . a  and  j  =  1 ,2,...,jj  we  define: 

^+ij  =  £jj  U 

(2.1.19)  Fy  =  Ejj  U  M1iH 

etij  =  M-jU£iiUMlj.1. 
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The  following  assumptions  on  the  saddle  point  (<j>*tO*)  can  now  be  stated: 

(2.1.20) 

(i)  The  fact  that  the  functions  <j>*  and  <l>*  are  piecewise  means 

there  is  a  regular  decomposition  associated  with  the  saddle  point 
(<j>*,0*)  on  £  with  the  property  and  O*  are  on  Ey  for  i  = 
1 ,2 . a,  and  j  =  1 ,2 . j j. 

(ii)  If  X0  e  F y  then  there  is  a  unique  optimal  trajectory  c* (XQ.t)  in  £j  for 
t0<t<  tjjj  where  tjjj  is  the  time  the  trajectory  reaches  Cj.  The  path 
is  never  tangent  to  sub-manifolds  /W[k,  k  =  j,..,jj  or  /V(1  jr. 

Berkovitz6  points  out  that  if  Xg  e  A/j-j . jr  then  X0  can  be  a  member  of 

several  £T y  for  several  values  of  i.  Then  condition  (2.1 .20)  (ii)  will  hold  for  each  of 
these  values  of  i.  The  time  derivative  of  the  optimal  trajectories  c*  will  in  general 
be  discontinuous  at  times  t,-^  when  the  trajectory  intersects  the  sub-manifolds 
Mlk,  k  =  j,..,jj_i .  This  is  because  either  <J>*  or  <D*  are  discontinuous  at  these  points. 

2.1.7.  VALUE  FUNCTION 

The  following  lemma  of  Berkovitz  will  be  needed  before  we  can  discuss 
the  properties  of  the  value  of  the  game.  This  lemma  describes  some  basic 
properties  of  an  optimal  trajectory,  and  its  intersections  with  manifolds  /l^k,  k  = 

i—.Ji-i* 


^Berkovitz  pg  6 
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LEMMA  2.1.1 7  Let  Xq  e  E^j,  tjk  =  tjk(XQ)  be  the  intersect  time  with  manifold 
/%,  xik  =  xik(xo)  be  the  mt©rsect  point  with  manifold  for  j-1  £  k  £  ]j.  Then  for 
j  s  k  <  jj,  c*(X0,t)  is  defined,  is  C^1 )  on  Et-^  x  [tj  ^.-j ,tjk],  and  satisfies: 

(2.1.21)  X*  =  f(X0,4»*(X),<l>*(X)) ;  X(0)  =  X0 

where  fortj  ^.-j  and  tjk  the  values  are  defined  as  limits  from  the  interior  of  £jk. 

Furthermore  for  j  s  k  <  jj  the  parameters  sk,  tjk,  and  X,k  of  manifolds  are 
CO )  functions  of  Xq  for  Xq  e  E^ jj®. 

proof:  The  optimal  controls  4>*(X)  and  0*(X)  are  by  assumption  on 
Ejk,  1  <  k  <  kj.  The  fact  that  Ejk  is  contained  in  a  bounded  subset  n  implies  there 
exists  continuous  extensions  <|>*jk(X)  and  0*jk(X)  on  a  region  £jkE  containing  E0^ 
of  the  restrictions  <)>*  |  Ejk  and  <X>*  |  Ejk. 

We  now  proceed  by  induction  on  each  of  the  regions  E^,  showing  the 
first  assertion  is  true  for  k  =  j  where  Xq  e  E^j.  Consider  the  differential  equation 
for  Xq  e  E^jj  on  Ey0. 

(2.1.22)  X*  =  f(X/ ||(X)fO*jj(X));  X(0)  =  X0 

The  existence  and  uniqueness  theorems  of  ordinary  differential  equations® 
guarantee  that  there  exists  cC1)  function  Ojj(XQ.t)  defined  for  a(XQ)  <  t  <  b(XQ) 
which  is  a  unique  solution  of  the  differential  equation  (2.1 .22). 


7Berkovitz  Lemma  1  pg  8 

8Recall  si  is  the  (n-1  )-dimensional  parameterization  of  A^k  via  2.1 .0. 
®Boothby  Theorem  4.1  pg  130 
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The  optimal  trajectory  is  assumed  to  exist  on  E^y.  This  along  with  the  fact 
that  (2.1.21)  agrees  with  (2.1.22)  on  Ey  implies  that  on  E^y,  oy(Xg,t)  =  c  (XQ,t) 
for  tj  j_i  <  t  <  ty,  where  ty  is  the  time  the  trajectory  intersects  My  and  tj  j_-j  is  the 
time  the  trajectory  intersects  Mg.- j  backwards  in  time.  The  first  assertion  holds 
for  k  =  j. 


We  now  show  the  second  assertion  is  true  for  k  =  j.  The  fact  that  si  and  ty 
are  C^1 )  functions  of  X0  follows  from  the  implicit  function  theorem,  o)j(XQ.tjj)  is 
C^1)  on  Eyg  x  (tj  j.i-S.tj  j+S)10,  hy  is  cC1)  (2.1.22),  the  condition: 

(2.1.23)  Oy(X0,ty)  -  hy(s)  =  0 

and  that  the  nxn  matrix 


(2.1.24) 


aq*(XQ,ty)  -ghy(si) 

9ty 


has  dimension  n  due  to  the  fact  that  the  optimal  trajectory  is  not  tangent  to  A/fy. 
Xy  is  a  C^1)  function  of  Xq  follows  from  the  fact  that  hy  is  a  C^1)  function  of  si,  si 
is  a  C^1 )  function  of  Xg,  and  Xy  =  hy(si).  The  second  assumption  holds  for  k  =  j. 

Assume  now  that  for  jj  >  k  >  j+1  ,  c*(XQ,t)  exists  and  is  on  E^k.-j. 
Also  that  sk_1 ,  tj  k_i ,  Xj  ^.-j  are  C^1 )  functions  of  Xq  e  E^j.  Let  a  e  E1^,  then  by 
similar  arguments  to  those  above,  there  exists  C^1 )  function  Ojj<(cx,t)  defined  for 
a(a)  <  t  <  b(a)  which  is  a  unique  solution  of  the  differential  equation: 

(2.1 .25)  X*  =  f(X,<j,*ik(X),<D*iK(X));  X(0)  =  a 


10The  fact  that  the  solution  oy(Xg,t)  to  (2.1.22)  exists  on  E^y  x  [tj  j.-j.tjj] 
implies  it  must  exist  on  some  open  region  containing  Etjj  x  [tjj.-j  ,tjj]. 
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Similar  arguments  to  those  above  imply  that  for  tjk_i  <  t  £  tjk, 

(2.1 .26)  c(Xi  k_,  .t-ti*., )  =  0|k(Xj )(.,  ,t-tj  k.i ). 

On  the  interval  tj  k_-j  <  t  <  tjk  we  also  have  by  the  optimality  of  the  trajectory, 

(2.1.27)  a(X0.t)  =  a(Xitk.1,t-ti>k.1) 

This  leads  to  the  conclusion  that  for  tj  k_i  <  t  <  tjk, 

(2.1 .28)  a  (X0,t)  =  Oj^Xj  k.!  .t-tj^ ) 

We  have  shown  that  Ojk  is  on  £E^jk  x  [tj  k.-j,  tjk],  and  by  assumption  Xjk_i 
and  tj  k.-j  are  C^1)  functions  of  X0.  This  implies  that  a*(X0,t)  is  d1)  on  E^j  x 

Pi.k-I*  *ikl- 

Last  we  must  show  that  sk_1 .  tj  k.1t  Xj^  are  cC)  functions  of  Xq  e  E~ j. 
Since  aik  is  d1)  on  EikE  x  (tj>k_i -5,tjk+5)  we  can  repeat  arguments  for  with 
(2.1 .23)  and  (2.1 .24)  where  we  simply  replace  j  by  k. 

*  * 

We  now  consider  the  properties  of  the  value  function  W(Xq)  =  P(Xo.<i>  )• 
The  following  lemma  of  Berkovitz  states  the  smoothness  properties  of  W(Xg). 
We  note  that  W  is  continuous  on  Ey. 

LEMMA  2.1 .2:11  W(X0)  is  a  ct1)  function  of  X0  e  E^j,  and  is  continuous  on  E. 

proof:  The  value  function  can  be  written  in  terms  of  the  contact  of  the 
terminal  manifold  C: 

(2.1 .29)  W(X0)  =  P(X0, <{>>*)  =  H(Xj  jj). 


^Berkovitz  pg  16 
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The  fact  that  H  is  C^1)  by  assumption,  and  Xj  jj  is  a  C^1)  function  of  Xq  e  E^j 
(Lemma  2.1.1)  implies  the  assertion. 

Since  W  is  continuous  on  E^y  we  need  only  show  for  continuity  on  £j  that 
for  every  X  e  /Wy  there  exists  a  sequence  from  each  side  of  Mjj  approaching  X 
whose  limits  are  equal.12  Let  Xq  s  £ y  such  that  a*(Xo,tjj)  =  X,  by  Lemma  2.1.1 
such  an  Xq  must  exist.  Along  any  trajectory  W(c*(XQ,t))  is  constant  implying 
W(o*(X0,ry))  =  W(c  (XQ.t+ip). 

The  assumption  that  the  payoff  function  is  single  valued  for  the  optimal 
trajectories  is  used  to  extend  the  continuity  of  W(X)  to  all  of  the  game  space  E. 

Let  X  e  A/*j  2,..., k*  and  <r*i(X,t) . o*x(X.t)  be  the  corresponding  optimal 

trajectories  in  each  region  £-|,...,£x.  Since  a*i(X,0+)  =  ct*2(X,0+)  =  ...  = 
a*K(X,0+),  and  W(X)  is  continous  on  all  £j  the  conclusion  holds. 

The  following  notation  needs  to  be  introduced.  If  Xq  e  Afy  then  Wx(Xq) 
=  lim  WX(X)  as  X  ->  X0  from  £y.  Likewise,  W+X(X0)  =  lim  WX(X)  as  X  X0 
from  Ej  j+i-  Lemma  2.1.2  guarantees  the  existence  of  these  one  sided  limits. 

One  more  lemma  is  needed  before  the  main  result  of  this  section  can  be 
handled.  This  lemma  states  the  properties  of  variations  from  the  optimal 
controls.  The  lemma  is  new  and  allows  the  extension  of  Berkovitz’s  work  to 
consider  only  the  family  of  autonomous  pure  strategies  for  autonomous  systems. 

LEMMA  2.1.3:  Let  <}>  e  j,  X0  s  Ey  and  R  >  0.  Denoting  the  trajectory 
corresponding  to  $,<!>*  by  o(t,X)  we  can  define  the  set: 

1 2Since  we  have  already  shown  continuity  on  E^y  we  do  not  have  to  show  this 
for  every  sequence. 
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(2.1 .30)  Nr(Xq)  =  {  X  :  W(X)  <  W(<j(R,X0))  }. 

Then  NR(X0)  satisfies  the  following  properties, 
i.)  NR(X0)  is  closed, 

(2.1.31)  ii.)  Xq  e  Nr(Xo). 

iii.)  If  X  «  Nr(Xq)  then  c*(t,X)  t  Nr(Xq). 


Define  for  <j>  e  a  corresponding  0R  by: 

r-  4>*(X)  if  X  £  NR(X0) 

(2.1.32)  L„X)ifX£NR(X0). 

Then  iv.)  <|>R  e  UM.  The  trajectory  corresponding  to  <|>R,«I>  is  denoted  by 
CTR(t,X). 


ILLUSTRATION  OF  NR(X0)  & 
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proof:  The  properties  i.)  and  ii.)  follow  easily  from  the  continuity  of  W(X). 
Property  iii.)  follows  from  the  optimality  of  the  strategies.  Assume  there  exists  an 
X  e  Np(Xo)comP  and  t  >  0  such  that  a  (t,X)  e  Nr(Xq).13  The  fact  the  we  are 
only  dealing  with  terminal  costs  implies  that  W(X)  =  W(c*(t,X))  hence  X  e 
NR(X0):  a  contradiction. 

We  now  turn  our  attention  to  property  iv.).  There  are  two  cases  to  be 
considered.  The  first  case  is  when  the  optimal  trajectory  never  leaves  Nr(Xq). 
Then  crR(t,X)  »  a(t,X)  for  t  >  0,  implying  the  trajectory  intersects  the  terminal 
manifold  in  finite  time  due  to  the  admissibility  of  <J>(X).  If  the  trajectory  leaves 
Nr(Xq)  at  time  T  then  aR(t,X)  =  c*(t-T,c(T,X))  for  t  >  T,  implying  the  trajectory 
intersects  the  terminal  manifold  in  finite  time. 

Isaacs'  equation  can  now  be  derived  in  the  following  theorem.  This  theorem  is 
new  and  considers  only  variations  of  autonomous  strategies. 

THEOREM  2.1.2:  On  E^j  the  following  equation  is  satisfied  for  all  <f>  €  U ^  and 

€  V14 

MinMax(Wx(X)f(X,<J>,0))  =  MaxMin(Wx(X)f(X,<j>,<D))  =  0 
O  <E>  <(> 

where  for  X  e  Mjj,  Mj  j.-j  the  equation  is  defined  as  a  limit  from  the  interior  of  Ey. 

proof:  Let  Xq  e  Ey,  and  <f>  e  Um  then  by  Lemma  2.1.3,  Nr(Xq)  and  <f>R  are 
^operly  defined  for  R  >  0.  The  assumption  that  <>*,0*  is  a  saddle  point  implies 
that: 

(2.1.33)  W(X0)  »  P(X0,(|>V)  £  P(X0,<j>R,O)  =  W(or(R,X0)) 

13NR(XQ)comP  is  the  complement  of  Nr(Xq). 

14WX  is  the  partial  derivative  vector  of  W  wrt.  X. 


23 


where  or  is  the  trajectory  corresponding  to  4>r,<I>  and  R  is  the  exit  time  from 
Nr(X0). 

The  condition  (2.1.33)  implies  that  W(Xq)  -  W(oR(R,Xo))  S  0  with  equality 
for  <j>  =  4>*.  Lemma  2.1.2  and  Lemma  2.1 .1  state  that  W  and  or  are  both  )  so 
the  mean  value  theorem  may  be  applied  twice  on  Nr(Xq)  using  the  fact  that  Xq  = 
^r(O.Xo)- 

(2.1.34)  W(X0)  -  W(or(R,X0)) 

=  -Wx(Xq)(cr(R,Xq)  -  Xq)  +  0(ctr(R,Xq)-Xq)2 
=  -Wx(X0)(a*R(0,X0)R  +  O(R)2)  +  O(R)2 

=  -wx(x0)f(x0,<KX0),<i>>o)>R  +  °(R>2 

<  0  (by  equation  (2.1.33)} 

Dividing  equation  (2.1 .34)  thru  by  R,  we  have  in  the  limit  as  R  ->  0: 

(2.1 .35)  Wx(X0)f(X0t<D(X0),O*(X0))  >  0  for  every  *  e  UM 

* 

with  equality  for  4>  =  0  . 

Similar  arguments  can  show  for  <J>  that: 

(2.1.36)  Wx(X0)f(X0,<MX0),O(X0))  <  0  for  every  <D  e  VM 

* 

with  equality  for  O  *  <l> . 

Combining  equation  (2.1,35)  and  (2.1.36)  we  have: 

(2.1.37)  Wx(Xo)f(Xo,0*(Xo),<D(Xo)) 

^  Wx(X0)f(X0,<|>*(X0),<D#(X0)) 

*  Wx(X0)f(X0,(|>(X0),<D*(X0)) 

for  every  <j>  e  U^,  ®  e  V^.  Equality  holds  for  4>  =  <j>  and  <1>  =  4>  simultaneously. 

Equation  (2.1 .37)  implies  the  assertion  of  the  theorem,  (2.1 .32)  on  ^j.  We 
now  extend  this  result  to  5^.  Without  loss  of  generality  assume  X  e  Afy  then 
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due  to  Lemma  2.1.2  and  the  fact  that  f  is  d1)  by  assumption,  the  limit  of 
equation  (2.1 .37)  as  Xq  ->  X  from  Ey  exists  and  implies: 

(2.1.38)  W-x(X)f(X)4>*ij(X),<D(X)) 

^W-x(X)f(X,0*ij  (X),0*jj(X)) 

<  w-x(X)f(X,<>(X),a>*jj(X)), 

where  the  continuous  extensions  of  the  controls  defined  earlier  are  used. 

The  last  question  to  be  tackled  in  this  section  is  the  continuity  of  Wx 
across  manifolds  My.  The  following  theorem  of  Berkovitz  handles  this  question  in 
the  case  both  d*  and  <D*  are  not  discontinuous  at  My. 

THEOREM  2.1 .3:15  If  <J>*  and  C>*  are  both  not  discontinuous  at  /tfy  then  Wx  is 
continuous  across  My. 

proof:  The  assertion  will  be  proven  if  we  can  show 
W"X(X0)  =  W+X(X0).  The  fact  that  W  is  d1 )  on  (Lemma  2.1.2),  and  My  is  a 
d1)  manifold  implies  (similarly  to  Lemma  2.1.1)  that  there  exists  a  d1) 
extensions  Wy(X)  on  a  region  EyE  containing  ^y  of  the  restriction  W  |  Ey.  Given 
arbitrary  X0  s  My  and  letting  x(r)  be  any  d1)  curve  lying  in  such  that  x(0)  = 
X0  we  have  w(r)  =  W(x(r))  =  Wy(x(r)).  Then  the  derivative  of  w(r)  at  r  =  0  is: 

(2.1.39)  w#(0)  =  WXjj(Xo)x*(0)  *  W'x(X0)x*(0). 

Similar  arguments  can  show  that, 

(2.1 .40)  w*(0)  =  W+x(X0)x*(0). 

This  leads  to 

(2.1.41)  (WX(X0)  -  W+x(X0))x*(0)  =  0. 


^Berkovitz  Theorem  1  pg  16 


25 


The  fact  that  x(r)  is  an  arbitrary  curve  and  /Wjj  is  a  (n-l)-dimensional 
manifold  implies  W+x(Xq)  =  W‘x(Xq)  or 

(W* x(Xq)  -  W+x(Xq))  is  nonzero  and  orthogonal  to  /Wjj  at  Xq.  Assume  the  latter 
and  without  loss  of  generality  that  C>*(X)  is  continuous  across  then  /Wjj. 


Analogous  to  Lemma  2.1.3  we  define  the  following  function  on  NR(Xg): 

r  <MX)  if  X  c  NR(X0) 

(2.1.42)  <t>ijR(X)  =  . 

<-*jj(x)  if  X  €  NR(X0), 

is  a  member  of  where  <}>  jj(X)  is  again  the  continuous  extension  of  the  control 


as  defined  earlier. 


The  results  of  Theorem  2.1.1  on  (2.1 .32)  imply  that: 

0  =  \N\(XqW0A^X0),*(X0)) 

(2  1  43)  S  ,R(X0).<t>'(X0)). 

0  =  W+x(X0)f(Xo,4*jj+1  (X0)  V(X0)) 

S  W+x(Xo)f(X0,<t>ijR(XQ),<l>*{X0)). 

Using  the  fact  that  <t>jjpj(X0)  =  <|>*jj(Xg)  and  combining  equations  of  (2.1 .42)  we  get: 

2  1  44  W  '  WWtXo^Xo).^))  £  0, 

(W-X(X0)  -  W+x(X0))f(X0,<|)\j+i  (Xg),O*(X0))  >  0. 

The  inequality  must  hold  in  equations  (2.1.43),  otherwise  due  to  the  assumption 
that  (W‘x(Xq)  -  W+x(Xq))  is  nonzero  and  orthogonal  to  /Wjj,  f(XQ,4>jj(X0),<I>*(Xo)) 
and  f(Xg,<t>j  j+i  (Xg),0*(Xg))  must  be  tangent  to  A4jj.  If  the  inequality  holds  then  on 
each  side  of  the  tangent  plane  of  A^j  at  Xg  the  trajectory  points  into  opposite 
sides  of  the  half  space,  an  impossibility.  This  implies  VVx(Xg)  =  W+x(Xg)- 
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2.1.8.  ADJOINT  EQUATIONS 

We  will  now  derive  the  adjoint  equations  following  Berkovitz.  The  fact  that 
the  ajoint  equations  are  derived  in  retro-time  from  the  terminal  manifold,  and  the 
optimal  trajectories  are  defined  only  in  forward  time  for  t  >  0  causes  us  to  first 
introduce  and  define  the  function  r(t,Xg).  Then  we  show  for  t  >0: 
aW(X) 

(2.1.45)  -  =  WX(X)  =  r{t,X0), 

ax 

with  X  =  a  (t.Xg). 

LEMMA  2.1.4:^  Let  Xg  e  E^y,  define  the  function  T(t,Xg)  as  follows.  On  each 
interval  [tj  ,  tjjJ,  j  <  k  <  jj,  T  satisfies  the  following  differential  equation: 

(2.1.46) 

r*  r 

ax 

af(o*(t,Xg),(D*(a*(t,Xo)),<D*(o*(t,Xo)))  a<t>V(t,Xg)) 

a<t>  ax 

af(o*(t,Xg),(|)*(o*(t,Xo)),cD*(a*(t,XO)))  ao*(o*(t,Xg)) 

ao  ax 

with  the  initial  condition  nt'j^.Xj^),  and  final  condition  nt+j,K-1  ’Xi,K-1  )•  The 
initial  conditions  on  each  interval  are  described  as  follows. 


^Berkovitz  Lemma  4  pg  19 
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If  k  <  jj  then  letting  r+iK  =  r(t+iK),  riK  is  defined  by  the  following  n  linear 
equations: 


(2.1.48) 


axik 

=  r+ik  — —  -<*  •(t+iK,x0) 

Bs 

Then  on  each  interval  [tj  k.i  ,tjk].  r  is  well  defined  and  a  continuous  function  of  t 
and  X0  for  0  <  t  and  j  <  k  <  jj. 

proof:  The  proof  will  be  by  induction.  We  first  show  the  lemma  holds  for  k 
a  jj.  The  matrix  on  the  left  hand  side  of  equation  (2.1 .47)  has  dimension  n  due  to 
the  non-tangency  condition.  This  implies  Tjj  is  uniquely  defined.  The  fact  that 


1 7These  are  known  as  the  transversality  conditions. 
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Xj  jj  is  a  C^1)  function  of  s,  and  Lemma  2.1.1  imply  that  (2.1.47)  uniquely  defines 
Fjj  as  a  continuous  function  of  Xq. 

Equation  (2.1.46)  defines  an  n-dimensional  linear  system  in  r.  The  fact 
that  the  matrix  on  the  RHS  of  equation  (2.1.46)  is  a  continuous  function  of  t  for 
tjjj.i  £  t  <  tj jj  implies  the  unique  solution  r(t,Xo)  exists  on  this  interval18.  The 
conclusion  then  holds  for  k  =  jj. 


We  now  show  that  if  the  conclusion  holds  for  k  +  1 ,  it  holds  for  k.  Equation 

(2.1.48)  uniquely  defines  since  the  non-tangency  condition  implies  the 
matrices  of  (2.1.48)  are  non-singular.  The  induction  assumption  implying  Fl'jk  is 
a  continuous  function  of  Xq,  along  with  Lemma  2.1.1  imply  that  Fx  is  a 
continuous  function  of  Xq.  Then  differential  equation  (2.1.46)  is  well  defined 
similar  to  k  =  jj  with  the  new  initial  condition  T(\\)  =  Fx  for  tj>X-i  <  t  £  tjX-  The 
unique  solution  r(t,X0)  exists  on  this  interval. 

THEOREM  2.1 ,4:19  Let  X0  €  5,j,  HLXq)  be  the  function  described  in  Lemma 
2.1.4,  if  X  =  a*(t,X0)  then  for  t  >  0: 

(2.1.49)  WX(X)  =  r(t,X0). 


proof:  Let  Xq  e  Ejj,  because  W(X0)  =  H(Xjjj),  and  from  Lemma  2.1.1 
parameter  si  is  a  C^1 )  function  of  Xq  we  have 


dH(Xjij) 

(2.1.50)  WX0(X0)=  - lL- 

9Xi,ji 


axi,ji  0S 

8s  8X0 


18Zadeh  and  Desoer,  1963 
19Berkovitz  Theorem  2  pg  21 
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Define  F  as  the  n  x  n  matrix  in  equation  (2.1.46),  so  that  we  can  write  r*  = 


Then  we  have 


(2.1.51)  WX0(X0)  = 


aH(Xiijj)  axjjj  ds 
9Xj  jj  9s  9Xq 


if1  ii.K+1  -  * 

da 

-  X  (r*  +  r  F)  —  dt 
k=j  J  9X0 


fIJ  9a 

-  (r*  +  r  F)  —  dt 


aH(Xiji)  9Xjji  _9s_ 

9Xj  jj  9s  9X0 


M 

J|  '  r  * 

X9a 

d  r  — 
k=j  J  9Xq 


d  r 


aH(Xjji)  axgi  _3s_ 

aXjji  3s  3Xq 


+ no,x0) 


3c  (O.X0) 


9a  (t  j  jj.Xg) 


X.  r+iK 

k=J 


(t+iK-xo)  ^  iKiXo) 

- -  -  Piu - 

9Xq  9Xq 
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Let  us  now  look  at  the  terms  in  the  summation  above.  Expanding  the 
partials  of  the  optimal  trajectory,  and  using  equation  (2.1 .48),  we  have: 

Jr1 


(2.1.52) 


3 


9a  (t+jK.X0)  do  (t'iK,X0) 

I^iK - - - rik  - 


9Xq 


9Xq 


Jr1 

I 

M 


Qr+iK«‘*(t+iK.Xo)-nko'*(fiK.X(,)]]  ^ 


9X 


Goa;  i*  9X;  — i  9S 

p-iK— '■£  -  rik— \  — 

»-■>  K  3s  j  3X0 


=  0. 


We  can  also  use  equation  (2.1.47)  to  show 


(2-1.53)  n 


9Xr 


=  Tii 


[j*  *(t+i,ji>xo) 


9t 


i  ,ji  ^xi,ji 

9s  9X, 


9X0 


Uo  — 

9X«  — 


9H(Xi  ji)  9Xj  jj  9s 


i.Ji 


9Xijj  9s  9X0. 

Combining  equations  (2.1.51),  (2.1.52),  and  (2.1.53)  along  with  the  known  fact 
that 

9o*(0,X0) 


(2.1.54) 


9Xr 


=  I  (nxn  identity  matrix), 


we  have  WXo(X0)  =  r(0,X0). 
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We  note  that  if  X  =  a*(x,X0)  then  o*(t,Xo)  =  c*(t-x,X)  for  t  >  x.  This  implies 
r(t,x0)  =  r(t*x,X)  for  t  >  X  due  to  equation  (2.1.46).  Hence  WX(X)  =  r(0,X)  = 

r(t,x0). 
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2.2.  GAMES  OF  KIND 

The  achievement  of  termination  (reaching  terminal  manifold  C)  is  the 
kernel  of  the  problem.  We  wish  to  determine  from  what  positions  in  the  game 
space  £  capture  can  be  guaranteed.  The  game  space  E  is  partitioned  into  the 
capture  region  and  escape  region.  The  surface  that  separates  these  two  regions 
is  termed  the  barrier  (B).  This  section  will  be  concerned  with  the  derivation  of  the 
equations  that  generate  the  barrier,  and  follows  Isaacs'  heuristic  treatment  found 
in  his  Rand  report. 

2.2.1.  FEASIBLE  CONTROLS 

At  each  X  e  £  we  assume  defined  a  set  of  feasible  controls  for  the  pursuer 
l/(X)  and  evader  V(X).  The  pursuer  is  free  to  pick  any  control  vector  p  e  U{X). 
The  evader  is  free  to  pick  any  control  vector  d>  e  V(X). 

2.2.2.  TERMINAL  MANIFOLD 

The  terminal  manifold  (C)  is  an  (n-l)-dimensional  manifold.  We  assume 
the  manifold  can  be  parameterized  by  Sji  tel,..,  n-1  and  the  coordinate  functions 
hj;  j  =  1,...,n  such  that, 

(2.2.1)  Xj  =  hj(si  ,..,sn_i );  j  =  1,..,n. 

The  condition  of  capture  is  defined  only  when  the  terminal  manifold  (C)  is 
penetrated.  The  condition  when  a  trajectory  X  reaches  C  without  penetration  and 
returns  to  £  will  be  considered  neutral.  This  neutral  condition  delineates  capture 
and  escape.  Defining  v  =  (yi ,...,  to  be  the  normal  to  C  at  x  extending  into  £, 
the  condition, 

n 

min  max  £  vj(x)f;(x,p,<J>)  >  0, 
p  <D  i=1 


(2.2.2) 


33 


implies  that  the  evader  E  can  prevent  immediate  termination  from  a  position 
sufficiently  close  to  C.  The  region  of  C  where  the  above  holds  is  referred  to  as 
the  non-usable  part  (NUP)  of  C.  The  opposite  condition 

n 

(2.2.3)  min  max  2  Vj(x)fj(x,<}>,<I>)  <  0 

<}>  O  i=1 

implies  that  the  pursuer  P  can  force  termination  for  positions  sufficiently  close  to 
C.  The  region  of  C  where  this  condition  holds  is  referred  to  as  the  usable  part 
(UP)  of  C.  The  neutral  condition  can  only  hold  for  trajectories  containing  a 
member  x  of  Csuch  that  the  condition, 

n 

(2.2.4)  min  max  2  vj(x)f;(xl$lO)  =  0. 

<j>  <X>  i=1 

The  region  of  C  where  this  condition  holds  is  referred  to  as  the  boundary  of  the 
usable  part  of  C  (BUP).  The  condition  for  the  BUP  is  given  on  an  (n-1)- 
dimensional  manifold,  hence  the  BUP  is  an  (n-2)-dimensional  manifold.  We  seek 
to  attach  to  the  BUP  a  surface  that  neither  the  pursuer  or  evader  can  penetrate 
and  that  separates  E  into  the  capture  and  escape  region.  We  begin  by 
discussing  these  semipermeable  surfaces  of  dimension  (n-1)  and  how  they  may 
be  attached  to  (n-2)  dimensional  manifolds. 

2.2.3  SEMIPERMEABLE  SURFACES 

Let  S  be  an  orientable  surface  separating  the  neighboring  space.  The  two 
directions  of  penetration  of  S  are  termed  the  P-direction  and  E-direction.  The 
"side"  of  the  surface  S  reached  after  penetration  in  the  P-direction  is  termed  the 
P-side.  The  definition  of  the  E-side  follows  naturally. 
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Definition:  A  surface  S  is  termed  semipermeable  at  X  when, 

i. )  There  exists  a  value  $  e  l/(X)  (the  controls  of  P)  such  that  when  Y  =  <{>*, 

no  vector  0  e  V{X)  causes  f(X,<>*1<I>)  to  penetrate  the  surface  S  in  the  E- 
direction, 

ii. )  There  exists  a  value  O*  e  V{X)  (the  controls  of  E)  such  that  when  Z  =  <D*. 

no  vector  0  e  U{X)  causes  f(X,0,O  )  to  penetrate  the  surface  S  in  the  P- 
direction. 

A  surface  S  is  termed  a  semipermeable  surface  (SPS)  if  for  every  x  e  S,  the 
surface  is  semipermeable. 

Theorem  2.2.1 :2Q  Let  S  be  a  smooth  surface  in  E  and  v(x)  =  (v-j(x) . vn(x))  be 

its  normal  vector  at  each  x  e  S.  Then  S  is  a  SPS  if 

n 

(2.2.5)  min  max  2  vj(x)f;(x,<J>,0)  =  0. 

<J>  O  i=1 

proof :  v(x)  is  oriented  such  that  it  points  in  the  E-direction  of  S.  Let  O*  be  the 
value  of  O  at  which  the  LHS  of  the  equation  above  assumes  its  maximum.  Then 
by  definition  for  every  feasible  <{>, 

n  .  n 

(2.2.6)  0  =  min  I  vj(x)fi(x,0I<D  )  <  I  vj(x)fj(x,0,O  ). 

<t>  i=1  i=1 

This  implies  there  is  no  value  of  0  that  causes  penetration  of  S  in  the  P-direction. 
The  opposite  case  follows  similarly. 


20lsaacs  pg  9.7 
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Theorem  2.2.2:21  Let  S  be  as  above  and  assume  that  on  an  open  set  N  relative 
to  S,  <j>  (x)  and  <T>*(x)  are  )  for  x  e  S.  Then  for  k  =  1  ,...,n  the  following  relations 
hold  on  A/; 


(2.2.7) 


dvK  n  afj(x,<j>  ,<D  ) 

-  =  -  I  Vj(x)  - . 

dt  i=1  3x^ 


proof:  The  Tietze-Urysohn  extension  theorem  implies  there  exists  a 
extension  of  vj(x),  <|>*(x),  0*(x)  from  N  to  E.  This  implies  that  the  partial  derivative 
of  (2.2.5)  with  respect  to  coordinate  x^,  k  =  1  ,...,n  exists  on  the  extensions  and  is 
given  by, 

n  3v,  n  3fs  ns  3f,  3<|>*j 

(2.2.8)  I  —  fj  +  I  v\  —  +  I  .1  Vj  — - 1 

i=1  3x|<  i=1  oXj^  i-1  j=1  8<|)j  3x^ 


n  r  3f;  30* ; 

+  I  I  Vj— - 1  =  0. 

i=1  j=1  30j  3x^ 


The  optimality  criterion  implies  that  the  two  double  summation  terms  are  zero. 
Looking  at  the  first  term  we  may  write, 


(2.2.9) 


n  s 
I  I 
i-1  J-1 


3fj  3<{>  j 
3<>j  3xk 


S  r  3 

=  I  - 

j=1  L  3cf>j 


=  0. 


If  <|>*(x)  is  an  interior  minimum  of  U(x)  then  the  first  bracketed  term  is  zero.  The 
alternative  case  of  an  exterior  minimum  implies  that  the  second  bracketed  term  is 
zero. 


21 
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Assume  that  there  exists  a  C^1 )  real  valued  function  G:E  -»  R  such  that  vj 
«  3G/3xj.  The  C1  nature  of  G  implies  that  9v^/9xj  =  dv-fix^  since  the  order  of 
partial  differentiation  of  G  makes  no  difference.  Then  we  may  write  that 


(2.2.10) 


n  3vs 

S  —  fi 
i=1 


y  3v,K  .  dvK 

i=1  9xj  '  dt. 


The  last  part  of  the  expression  follows  from  the  second  summation  which  is  the 
time  derivative  of  along  a  trajectory  corresponding  to  the  optimal  controls  O* 
and  <)>*  through  x.  Substitution  of  (2.2.10)  and  (2.2.9)  into  (2.2.8)  leads  to  (2.2.7) 


page  35.  This  leads  to  the  reverse  path  equations  (RPE)  where  t  =  tf  - 1: 


(2.2.11) 


dvv  n  afj(x,<j>*,<D*) 

-  *  I  VJ(X)  - 

dt  i=1  8xj< 


'"'IS,  *  * 

—  =  -  & ) 
dt 


Theorem  2.2.3:22  Let  D  be  an  (n-2)-dimensional  manifold  in  E  parameterized  by 

s-| . sn_2  with  Xj  =  hj(s-|  ,...,sn.2);  i  =  1 . n  the  smooth  coordinate  functions  of  D. 

If  the  normal  vector  v(s-| ,...,sn.2)  =  {^■j(si,...,sn.2),..,Yn(s1l...,sn.2)}  to  D  is  such 
that  on  D, 

(2.2.12)  I  yj(x)fi(x,<j)*10*)  =  0, 
i=1 


with  Xj(t,s1,...,sn.2)  and  V|(x,si,—sn-2)’  ■  =  integral  curves  of  the  RPE  with 
initial  conditions  yj.hj,  then  Xj(t,s-j,...,sn_2)  is  the  parametric  representation  of  a 
SPS  S  containing  0. 


^Isaacs  pg  9.13 
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proof:  We  first  show  that  the  v k  derived  through  the  RPE  is  the  normal  to 

the  surface  S.  Defining  the  functions  T^(x,si . sn_2),  k  =  1 . n-2  by, 

n  9xj 

(2.2.13)  TK  =  X  Vj(x)  - , 

i=1  dsK 

we  note  T^(0,s-| . sn.2)  =  0  since  this  is  exactly  the  normality  condition  on  D. 


Taking  the  derivative  of  with  respect  to  x  we  have, 


dTK 

n 

3^Xj 

n 

3xj 

3vj(x) 

(2.2.14)  = 

I 

^j(x)  - - 

+ 

I 

dx 

i=1 

9x3s^ 

i=1 

9sk 

9x 

Noting  that, 

3^x; 

3fj 

n 

9fj 

9xj 

s 

9fj  9<j)  j 

(2.2.15) 

— 

I 

-  I 

3x9sj^ 

asK 

j=1 

9xj 

9sk 

j=1 

3<t>j  9xk 

r  9f;  30* ; 

-X  — 1 - -J. 

j=1  30j  3xk 

The  optimality  criterion  implies  that  the  last  two  terms  of  the  RHS  of  equation 
(2.2.15)  are  once  again  zero  inferring, 


d^x:  n  3f:  9x; 

(2.2.16)  - -  =-X - -. 

9x9sk  j=1  d*j 


Substituting  (2.2.15)  and  (2.2.16)  into  (2.2.14)  we  conclude, 

dTi<  n  n  3f;  3x;  n  n  9x;  9f; 

(2.2.17)  —  =-  X  Ivj(x)  — -  — -  +  X  I  - v'j(x)— ! - 

dx  i=1  j=1  9xj  9s  k  i=1j=1  9S|<  3xj 

=  0. 

The  fact  that  T^  =  0  on  D  and  (2.2.17)  imply  T«  =  0  on  S. 
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We  now  define  the  function  Q(x,Si,...,sn.2)  by, 

n  „  * 

(2.2.18)  Q  =  I  vj(x)fj(x,<j>  .O  )■ 

i=1 

Taking  the  derivative  of  Q  with  respect  to  x  we  have, 

dQ  n  8f;  n  3vs 

(2.2.19)  -  =  I  Vj  -  +  I  f;  -  =0, 

dx  i=1  3x  i=1  8x 

by  similar  arguments  to  those  used  earlier.  Since  Q  =  0  on  D,  Q  =  0  on  S. 

Let  p(w)  be  a  smooth  curve  on  S  passing  thru  a  point  x  e  Ssuch  that  p(0) 
=  x,  then  the  tangent  vector  components  for  i  =  1 . n  to  p(w)  at  w  =  0  are, 

n-2  3x:  3pj  8p- 

(2.2.20)  I - -  +  fj  — -  . 

j=1  3Sj  9w  9w 

The  pt,  pj  are  smooth  real  valued  coordinate  functions  of  p(w)  such  that  Sj  =  pj(w) 
and  x  =  pT(w).  Taking  the  inner  product  of  the  tangent  vector  of  p(w)  with  the 
vector  v  we  have, 

n  r  n-2  3x;  3p;  9p- 

(2.2.21)  I  |  X  —  — L  +  f;  —  1  v) 

i=1  L  j=i  9s:  9w  3w  -I 


n-2  3p  r  n  3xj  ^  r  "  i 

=  X  —  I  -  vj  + -  XfjVj 

i=1  8w  L  i=1  3s;  J  3w  L  i=i  '  J 


dPx  r  " 


rTii  +^i[q] 

i=1  8w  l  J  J  3w  L  -I 
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This  implies  that  the  vector  v  is  normal  to  the  surface  S.  Lastly  S  is 
semipermeable  because, 

n  n  .  . 

(2.2.22)  min  max  I  vj(x)fj(x,<|>,0)  =  I  Vj(x)f;(x,<j>  ,G>  ) 

0  <D  i=1  i=1 

=  Q  =  0. 

2.2.4  BARRIERS 

Clearly  the  barrier  B  that  separates  the  capture  and  escape  region  of  E  is 
a  semipermeable  surface.  The  barriers  B  that  we  will  be  interested  in  will  be 
attachable  to  the  terminal  manifold  C.  This  corresponds  to  the  case  where  the 
Barrier  is  a  function  cf  the  maximum  vehicle  dynamics.  It  is  possible  for  a  barrier 
to  exist  that  does  not  attach  to  C.  The  case  of  an  intercept  problem  where  fuel  is 
limited  is  an  example  of  such  a  case.  Assuming  the  pursuer  is  faster,  fuel  limited, 
and  equally  maneuverable  as  the  evader,  it  is  obvious  that  a  barrier  exists 
corresponding  to  fuel  exhaustion  that  prevents  capture.  We  will  not  be 
concerned  with  these  type  of  barriers  here,  as  traditional  separation  principles 
allow  us  to  deal  with  these  type  of  constraints. 

The  boundary  of  the  usable  part  (BUP)  has  been  shown  to  be  an  (n-2)- 
dimensional  manifold  such  that  BUP  =  all  x  €  C  such  that, 

n 

(2.2.23)  min  max  X  ^(x)fj(x,0,O)  =  0, 

0  <J>  j=1 

where  Vj(x)  is  the  normal  to  x  e  C  pointing  into  E.  The  x  e  BUP  and  v,  serve  as 
the  initial  conditions  to  the  RPE, 


dv^K 

dx 
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(2.2.24) 

dxk 


dx 


n 

=  I  v)(x) 
i=1 


3xk 


=  -  fk(x,<t)*.<l>*). 


where  4>  and  O  are  the  continuous  solutions  on  E  to 

n 

min  max  £  Vj(x)fj(x,0,cl>)  =  0. 

<\>  <X>  i=1 


The  theorem  of  the  last  section  implies  then  that  the  (n-l)-dimensional  manifold 
generated  by  the  RPE  and  initial  conditions  on  C  is  a  semipermeable  surface. 

There  are  a  number  of  important  observations  about  the  geometry  of  the 
barriers.  The  first  is  that  they  may  end  abruptly  when  a  solution  no  longer  exists 
to  (2.2.24)  for  the  controls,  or  at  a  curve  of  discontinuity  of  the  optimal  controls. 

The  BUP  may  not  be  connected  (actually  the  most  common  case).  This 
means  that  there  are  a  number  of  barriers  connected  to  the  terminal  manifold. 
Each  of  these  barriers  is  attached  to  a  connected  submanifold  of  dimension  (n-2) 
of  BUP.  The  way  these  barriers  intersect  determines  the  capture  and  escape 
regions.  If  the  barriers  and  UP  of  C  form  the  boundaries  of  closed  regions  of  E 
then  the  interior  of  these  regions  are  the  capture  region  if  the  optimal  controls 
agree  for  each  barrier  at  the  intersection  point.  It  may  be  that  the  capture  region 
may  not  be  connected,  but  consists  of  a  number  of  connected  subregions.  The 
escape  region  consists  of  that  part  of  E  that  is  not  part  of  the  capture  region  or  its 
boundaries.  The  boundary  between  these  two  regions  of  course  consists  of 
portions  of  the  barriers. 

The  barriers  may  in  fact  not  meet  or  form  the  boundary  of  a  closed  region 
of  E.  In  this  case,  assuming  no  other  type  of  barriers  not  connected  to  C,  the 
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playing  space  E  serves  as  the  capture  region.  The  barriers  become  a  singular 
surface  that  trajectories  must  swerve  around  to  reach  the  UP  of  C. 
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3.  SELF  MODIFYING  SYSTEMS 

This  section  explores  systems  and  modelling  techniques  that  fall  outside 
the  classical  techniques.  Mesarovic  formalized  the  traditional  concepts  for 
systems  in  which  thrre  was  a  fixed  set  of  objects,  and  a  fixed  set  of  relations 
between  the  objects.  The  type  of  systems  we  are  exploring  have  the  property 
that  the  system  is  self  modifying  in  the  objects  and  relations.  These  type  of 
systems  we  term  semantic  systems.  It  is  important  to  understand  how  a  system 
such  as  air-combat  can  evolve  before  an  attempt  is  made  to  the  control  the 
system's  evolution.  Semantic  systems  will  be  shown  to  be  a  natural  extension  of 
Mesarovic's  systems.  Simulation  techniques  for  investigating  the  complexity  of 
semantic  systems  will  also  be  explored. 
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3.1.  GENERAL  SYSTEMS 


A  "system"  can  loosely  be  defined  as  an  aggregation  or  assemblage  of 
objects  joined  in  some  regular  interaction  or  interdependence.  A  system  "entity" 
is  an  object  of  interest.  Properties  of  "entities"  are  "attributes".  "Activities"  are 
processes  that  cause  changes  in  the  "system".  We  start  by  defining  the 
traditional  mathematically  rigorous  definition  of  a  general  system  as  originated  by 
Mesarovic  (5). 

Definition:  Let  T  be  an  arbitrary  linear  ordered  index  set,  a  formal  object  Jy; 
j  e  {1,2,...,n},  is  a  set  whose  elements  ei(t);  t  e  T,  are  called  the  values  of  the 
object  |y. 

Note:  The  index  set  T  may  be  of  finite,  countable  or  uncountable 
cardinality.  The  case  of  Card(T)  =  1  might  correspond  to  the  constrained 
extremization  system,  while  card(T)  =  R  and  card  (T)  =  N  would  correspond  to 
continuous  and  discrete  time  systems  respectively.  We  now  define  general 
systems  and  the  more  structured  case  of  input-output  systems. 

Definition:  (Explicit)  A  general  System  Es  is  generated  by  the  Cartesian 
cross  product: 


I  =  Il  *l2x-*In 

and  the  relation  R  on|  such  that  Es  is  a  proper  subset  of  £  (i.e.  e(t)  e  Es  iff 
Rfe’OI.e^t) . en(t))). 
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Definition:  Let  I  =  {1 ,2 . n}  and  lx»  ly  be  sets  forming  a  partition  of  I.  The 

set  X  =  x{Ej:  j  e  lx}22  is  termed  the  input  object,  while  the  set  Y  =  x{g:  j  e  ly}  is 
termed  the  output  object.  The  system  Es  is  then 

Esc  X  x  Y 

and  is  referred  to  as  an  input-output  system. 

Input-output  systems  are  defined  in  terms  of  relations.  This  means  that 
there  may  be  a  set  of  outputs  for  a  given  input.  If  we  apply  a  particular  input  to 
an  actual  system  we  know  we  will  get  out  a  single  output,  not  a  set  of  outputs. 
This  means  that  there  is  something  still  missing  from  our  definition:  the  concept 
of  state  that  determines  what  the  output  should  be  for  a  particular  input. 

Definition:  Given  a  general  system  Es,  let  C  be  an  arbitrary  set  and  F  a 
function,  F:(C  x  X)  Y,  such  that  (x,y)  e  Es  iff  there  exist  c  e  C  such  that  F(c,x) 
=  y.  C  is  then  a  global  state  set  whose  elements  are  the  global  states.  F  is  a 
global  systems-response  function. 

It  should  be  noted  that  a  state  object  is  not  equivalent  to  having  a  state 
space.  A  state  space  exists  only  for  dynamical  systems  in  which  the  semigroup 
property  (among  others)  is  in  effect.  Dynamical  systems  are  a  specific  type  of 
family  of  general  systems. 

The  problem  with  this  explicit  specification  of  a  general  system  is  that  both 
B  and  H  may  be  infinite  sets  which  may  be  only  specified  through  induction.  Thus 
^  may  be  defined  by  an  initial  set  of  generating  elements  E  and  a  set  of  syntactic 
rules  (S)  to  generate  syntactically  correct  elements  from  previously  generated 

22Cartesian  product 
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elements.  The  system  relations  B  may  also  specified  inductively.  A  set  of 
primary  relations  (R)  is  given,  and  a  set  of  axioms  (A)  is  specified  that  defines 
how  members  of  R  are  combined  to  form  B-  This  leads  to  the  implicit  definition  of 
a  system. 

Definition:  (Implicit)  A  general  system  is  a  quadruple  (E,S,T,A). 

We  will  now  introduce  the  formal  concept  of  a  classical  system  model  as 
defined  by  Mesarovic  (5).  Conceptually  a  model  should  be  a  system  that 
simplifies  the  system  being  modelled  such  that  "important"  properties  are 
incorporated  while  other  "unimportant"  properties  are  neglected. 

Definition:  Let  Es  c  X  x  Y,  Es'  c  X’  x  V  be  two  general  systems,  with  the 
functions  hx:X  — »  X',  hy:Y  -»  Y'.  The  function  h  =  hx  x  hy:  X  x  Y  — »  X'  x  Y'  is 
called  a  relational  homomorphism  from  Es  to  Es'  if  and  only  if: 

(x,y)  e  Es  ->  (hx(x),hY(y))  e  Es’ 

if  and  only  if  h  is  a  bijective  relational  homomorphism  such  that: 

(x'.y')  e  Es'  (hx"1(x),hy‘1(y))  e  Es 

then  h  is  a  relational  isomorphism. 

Es'  is  a  structural  model  for  Es  if  and  only  if  there  exists  a  relational 
homomorphism  h:  Es  ->  Es'  and  the  two  systems  are  structurally  equivalent  if 
and  only  if  h  it  a  relational  isomorphism. 

A  general  system  model  is  then  an  quintuple  (E.S.T.A.h). 
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3.2.  SEMANTIC  SYSTEMS 


The  paradigm  for  analysis  and  synthesis  of  systems  in  the  past  have 
assumed  Mesarovic's  definition  of  systems  and  models.24  There  are  many 
complex  systems  where  this  definition  falls  short.  Many  complex  systems  are 
self  modifying.  This  modification  takes  place  in  an  evolution  of  the  generating 
relation  R,  the  system  objects  g,  or  the  order  of  the  system  n.  Air  combat  is  one 
system  that  exhibits  these  properties.  The  system  components  consist  of 
aircraft,  missiles,  SAMS,  and  ground  targets.  The  destruction  of  an  object  or 
launching  of  missiles  modifies  the  systems  order  and  relations. 

Let  Eg'  be  a  Mesarovic  general  system  defined  by  the  relation  R'  such  that 
Eg'  c  x{Ej‘:  j  e  Jj}.  The  set  E  =  {Eg':  i  e  1}  is  the  set  of  system  generation 
components.  This  set  defines  the  types  of  feasible  components  with  which  we 
are  concerned,  and  multiple  copies  may  exist.  The  set  E  for  the  air  combat 
problem  would  consist  of  airplanes,  missiles,  SAMS,  and  ground  targets. 

We  now  go  about  defining  the  feasible  partial  relations  that  coordinate 
components  of  the  system.  A  set  of  feasible  relations  Ft,  for  each  i  e  I  is  defined. 
The  domain  for  each  re  Ft  is  of  the  form  x{EjKa:  j  e  J^a,  J^a  c  J^,  Ka  e  I,  a  e 
A}.  The  range  for  each  re  Ft  is  of  the  form  x{Ej':  j  e  J,  J  c  J,}.  The  index  set  A 
defines  the  mix  of  components  that  effects  a  component  of  type  i,  Kg  indicates 
the  type  of  component  for  a  e  A,  and  J«a  indicates  what  objects  from  a 
component  of  type  Ka  effect  a  component  of  type  i.  The  index  set  J  indicates 
what  objects  from  a  component  of  type  i  are  effected.  The  case  of  input-output 
systems  has  the  domain  restricted  to  output  objects,  and  the  range  restricted  to 


24The  practitioner  may  not  have  been  cognizant  of  this  fact. 


47 


input  objects.  It  should  be  noted  that  a  component  of  type  i  may  be  effected  by 
multiple  components  of  type  j  though  the  object  sets  from  the  multiple 
components  of  type  j  may  be  different.  The  set  R  =  Uj6  j  &  defines  the  set  of 
feasible  relations  for  the  system. 

A  semantic  state  consists  of  the  tuple  (S,Rg,M)  where 
S  =  {Es'm:  im  e  I,  m  e  M}, 

RS  -  'm  €  I*  type(r^)  e  fl*m,  m  e  M }, 

and  M  defines  the  mix  of  components  for  the  system.  The  notation  i™  is  used  to 
indicate  a  feasible  relation  for  a  component  of  type  im  e  I  from  f4m  for  m  €  M. 
We  note  that  two  relations  for  component  m,  r*m  and  r™  may  be  of  the  same 
type  but  are  considered  different  if  the  components  assigned  to  the  domain  are 
assigned  differently  or  are  different.  We  call  M  the  semantic  order,  S  the 
semantic  form,  and  Rg  the  relational  state.  The  semantic  form  S,  is  fixed  for  a 
fixed  M,  I.E.  a  component  may  not  change  its  type  during  the  course  of  its 
existence.  Alternatively,  Rg  is  not  fixed  for  fixed  M,  a  component  may  change  its 
relation  to  other  components  as  the  need  arises.  A  change  in  (S,Rg,M)  is  termed 
a  semantic  event. 

The  semantic  order  M  may  evolve  in  two  manners,  reproduction  or 
attrition.  Letting  M'  denote  the  new  semantic  order,  then  M  c  M*  indicates  a  pure 
reproductive  semantic  event.  Similarly,  MdM’  indicates  a  pure  attrition  semantic 
event.  A  semantic  event  may  occur  such  that  M  =  M',  implying  that  only  the 
relation  state  has  changed.  This  type  of  semantic  event  is  termed  a  relational 
semantic  event.  The  classification  of  a  semantic  event  as  pure  is  not 
guaranteed,  an  event  of  mixed  origin  is  the  normative  case. 
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The  relational  state  Rg  is  also  subject  to  change.  Obviously  a  change  in 
semantic  order  will  force  a  change  in  Rg.  The  relational  state  may  also  change 
while  the  semantic  order  remains  fixed.  This  change  would  take  place 
dependant  upon  the  traditional  global  state  and  systems-response  of  the 
component  types  indicated  by  the  semantic  form  S.  The  purpose  of  S  is  to  carry 
the  traditional  Mesarovic  information. 

The  values  of  the  objects  of  the  components  of  S  may  be  varying 
depending  upon  classification  (I.E.  dynamical,  decision)  of  the  general  system 
type  of  the  component.  Rg  coordinates  the  components  of  S,  hence  Rg  has  an 
influence  on  the  components  of  S.  S  may  be  continuously  changing  with  respect 
to  object  values  of  the  components,  but  the  member  components  only  change  at 
semantic  events. 

We  will  now  describe  the  constraints  under  which  a  semantic  system  can 
evolve.  Each  component  type  Es',  i  e  I,  has  a  set  of  reproductive  rules  Fj  that 
may  be  empty.  Members  of  Fj  consist  of  the  ordered  tuples  (Esi.flj.Cpi)  such 
that  j  e  I,  Esi  e  E,  Rj  c  R i,  and  Cpi  c  Es'.  The  tuple  indicates  that  the 
component  of  type  Es1  may  reproduce  a  component  of  type  Es*  with  an  initial 
relation  re  Rj  if  the  conditions  for  reproduction  CrJ  are  met  by  component  of 
type  Es1.  The  set  Rj  indicates  the  set  of  feasible  relations  from  which  the  new 
component  can  be  initialized.  The  set  Rj  may  not  be  empty.  The  set  CpJ 
indicates  the  conditions  on  the  component  of  type  Es1  under  which  the  new 
component  may  be  reproduced.  Clearly  if  CrJ  is  empty  then  reproduction  of  the 
new  component  is  not  possible,  Cpl  =  Es1  implies  that  reproduction  is  always 
possible. 

We  have  so  far  described  the  type  of  reproduction  that  may  take  place  for 
a  member  of  the  system  generation  component  set.  We  now  describe  the 
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consistency  conditions  that  express  a  component  can  not  reproduce  another 
component  if  there  does  not  exist  a  initial  feasible  relation  for  the  new  component 
whose  domain  consists  of  component  types  currently  in  the  system.  A 
component  me  M  of  type  im  may  reproduce  a  component  of  type  Esi  when 
(Esi.flj.Cpi)  e  Fjm  only  if  there  exists  subset  M  c  M  with  m  e  M,  r  e  fij,  and  a 
correspondence  C:A  ->  M  such  that  Ka  =  iQ(a)  for  every  a  €  A.25  This  simply 
states  that  reproduction  can  only  take  place  when  thtr;  exists  an  initial  feasible 
relation  whose  domain  can  be  well  defined  by  an  appropriate  assignment  of 
components  currently  in  the  system. 

The  attrition  rules  for  a  semantic  system  describe  how  components  of  M 
may  be  removed  from  the  system.  There  are  two  classifications  of  attrition  rules 
for  semantic  systems.  The  first  classification  consists  of  the  rules  of  self  attrition 
which  describe  the  mechanism  by  which  a  component  can  cause  itself  to  be 
removed  from  M.  The  self  attrition  rules  for  a  component  of  type  Es'  are  defined 
by  the  set  of  self  attrition  rules  C§a  c  Es*.  CgA  specifies  the  conditions  under 
which  a  component  ceases  to  exist  independent  of  the  action  of  all  other 
components  of  the  system.  We  note  that  CgA  must  b®  a  proper  subset  or  the 
component  could  not  possibly  exist.  It  is  also  possible  that  CgA  is  empty.  Self 
attrition  is  closely  related  to  concepts  such  as  stability,  and  life  expectancy.  An 
aircraft  that  goes  unstable  will  cease  to  exist  once  it  hits  the  ground.  A  missile 
has  a  life  expectancy  related  to  the  bum  time  of  its  motor. 

The  other  classification  for  attrition  in  a  semantic  system  is  attrition  of  one 

component  caused  by  another.  This  type  of  attrition  is  termed  causative  attrition. 

Causative  attrition  is  defined  for  each  component  type  Es',  i  e  I  by  the  set  of  rules 

25Ka  and  A  were  defined  previously  for  the  definition  of  the  domain  of  a  feasible 
relation. 
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Gj  which  may  be  empty.  A  member  of  Gj  is  the  ordered  tuple  (Esi.CQ^i)  such 
that  j  e  I,  Esi  e  E,  and  c  Es'  x  Esi.  The  tuple  indicates  that  the  component 
of  type  Es'  will  cause  a  component  of  type  Esf  to  be  removed  if  the  conditions  for 
attrition  Cq^  are  met  by  Es'  and  Esi.  Cq^  must  be  a  proper  subset  to  have 
significant  meaning.  Cq/^J  empty  implies  that  a  component  of  type  Esi  may  not 
be  removed  by  a  component  of  type  Es‘.  The  case  =  Es'  x  Esi  would 
express  that  the  existence  of  a  component  of  type  Es'  implies  the  removal  of  a 
component  of  type  Esi.  An  important  point  of  semantic  systems  is  that  it  may  not 
be  possible  for  a  component  of  type  Es'  to  remove  a  component  of  type  Esi 
directly,  but  a  component  of  type  Es*  may  be  able  to  reproduce  a  component  of 
type  Es^  that  can  remove  the  component  of  type  Esi. 


3.3.  SEMANTIC  CONTROL 


Previously  we  have  given  a  description  of  self  modifying  systems  that 
have  been  termed  semantic  systems.  A  component  of  type  Eg'  (i  e  I)  of  a 
semantic  system  had  the  capability  to  choose  a  relationship  r  €  f?  to  other 
components.  A  component  also  had  the  capability  to  reproduce  other 
components  according  to  the  set  Fj  or  cause  attrition  of  other  components 
according  to  the  set  G\.  The  choice  of  a  components  relation  to  other 
components,  to  reproduce  a  component,  or  remove  a  component  is  a  form  of 
control  over  the  semantic  state  of  the  system.  This  type  of  control  is  termed 
semantic  control. 

Each  component  or  group  of  components  is  assumed  to  have  a  directive. 
The  directives  specify  the  purpose  of  a  component.  Groups  of  components  with 
the  same  directive  may  cooperate.  These  directives  for  the  air  combat  problem 
correspond  to  missions.  A  group  of  fighter  bombers  for  the  air  combat  problem 
may  be  assigned  the  mission  of  destroying  a  tactical  site,  they  may  cooperate  in 
achieving  this  directive  either  in  bombing  runs  or  handling  interceptors. 

The  semantic  control  problem  for  a  component  i  e  I  is  to  determine  the 
"best"  r  e  Ff  dependant  upon  the  semantic  order  M  that  allows  the  component  its 
"best"  attempt  to  achieve  its  directive.  It  should  be  noted  that  a  component  may 
only  be  able  to  achieve  its  directive  by  reproducing  another  component.  The 
"best"  r  €  Ft  in  this  case  is  one  that  brings  the  component  to  a  condition  of  Fj  for 
reproduction.  This  means  for  the  air  combat  problem  an  aircraft  chooses  a 
strategy  (r  e  Ff)  that  brings  the  aircraft  to  a  missile  firing  point  with  its  target. 
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The  set  of  all  semantic  states  (S,  Rg,  M)  clearly  can  be  an  uncountable 
set.  A  component  m  e  M  may  also  only  have  limited  or  no  knowledge  about 
other  m'  e  M  (stealthing,  limited  sensor  range).  Clearly,  it  is  not  possible  to 
anticipate  every  situation  or  even  to  determine  if  a  specific  situation  exists.  Still  a 
component  must  try  to  meet  its  directive  as  best  it  can  with  limited  knowledge 
and  understanding  of  specific  semantic  states.  The  difficulty  of  designing  a 
semantic  controller  is  one  reason  humans  are  still  in  the  loop  in  many  real  world 
systems  that  are  in  reality  semantic  systems. 

Semantic  control  contrasts  in  many  essential  details  with  classical  control. 
Classical  control  assumes  that  the  designer  can  analyze  the  plant  and  generate 
the  control  laws  a  priori.  The  underlying  assumption  is  that  plant  and  control 
laws  are  fixed.  This  implies  that  the  plant  is  a  Mesarovic  type  system  in  which 
the  set  of  formal  objects  j  €  {1,2,..,n}  is  fixed  and  that  the  generating  relation 
B  is  also  fixed. 

The  semantic  control  system  consists  of  a  set  of  classical  control  systems 
which  represent  choices  for  r  e  Ft™  for  components  m  e  M  under  control  of  the 
semantic  control  system,  along  with  a  designer  as  depicted  in  figure  3.1 . 
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FIGURE  3.1 

SEMANTIC  CONTROL  SYSTEM 


The  designer  may  be  automated  or  semi-automated  and  consists  of  three 
blocks: 

1 .  System  Identifier, 

2.  Goal  Selector, 

3.  Control  System  Adapter. 

These  blocks  are  genetically  referred  to  as  correlators.  Each  correlator  searches 
its  specific  search  space  for  the  best  alternative  according  to  the  current  data  and 
specific  search  knowledge. 

The  purpose  of  the  System  Identifier  correlator  is  to  estimate  the  current 
semantic  state.  The  identifier  correlator  consists  of  an  inference  engine  and 
knowledge  base.  The  knowledge  base  consists  of  a  set  of  structural  models  (1^) 
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of  Mesarovic  form.  This  set  of  structural  models  may  be  explicitly  defined,  or 
implicitly  defined  via  a  generating  set  and  an  algebra  of  construction.  Typically  a 
member  of  1^  has  subsystems  that  are  structural  models  for  components  of  E. 

The  inference  engine  for  the  System  Identifier  must  choose  the  best 
structural  model  from  the  knowledge  base  and  estimate  the  state  object  from  the 
state  object  set  for  the  chosen  structural  model.  There  may  be  more  components 
in  the  semantic  order  M  than  any  of  the  structural  models  in  the  knowledge  base 
can  account  for.  The  inference  engine  must  choose  which  of  the  components  of 
M  will  be  accounted  for  in  the  structural  model  chosen. 

The  System  Identifier  for  the  air  combat  problem  for  a  specific  aircraft 
component  determines  how  the  aircraft  component  itself  should  be  modeled 
along  with  enemy  aircraft.  Assuming  the  knowledge  base  can  only  handle  one- 
on-one  encounters  the  inference  engine  might  choose  to  model  the  aircraft  itself 
as  an  infinitely  maneuverable  fixed  velocity  two  dimension  model.  An  adversary 
might  be  modeled  as  a  limited  maneuverable  but  faster  two  dimension  model. 
The  identifier  would  also  have  to  determine  best  approximations  of  the  state  of 
this  lower  dimension  model  that  approximate  the  much  higher  order  aircraft 
components. 
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FIGURb  3.2 


SYSTEM  IDENTIFIER 


The  Goal  Selector  correlator  determines  a  trajectory  in  the  semantic  state 
space  based  on  the  structural  model  identified  by  the  identifier.  This  trajectory  is 
chosen  to  allow  survivability  and  reliability  according  to  the  component's  directive. 
It  may  not  be  possible  to  meet  this  semantic  goal  exactly. 

The  Goai  Selector  for  the  air  combat  problem  solves  the  differential  game 
specified  by  the  System  Identifier  correlator.  The  results  are  the  optimal 
trajectories,  barriers  and  controls.  These  controls  are  not  the  same  controls  as 
those  needed  by  the  aircraft,  due  to  the  fact  that  the  model  chosen  by  the  Svstem 
Identifier  is  a  simplification  of  the  actual  encounter. 
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FIGURE  3.3 


New  Goal  Request 


Observable 


Inputs  X 


Model  Trajeotory 
Model  Controls 


Goal  Selector 

Model  Analysis 
Model  Trajectory 
Model  Controls 
Reproduction  Gcal 
Attrition  Goal 


Observable 


Outputs  V 


New  Model  Request 


Model 


State 

Attached  Components 


Goal  Selector 


The  Control  System  Adapter  determines  the  control  strategy  that  will  meet 
the  semantic  goal.  This  control  may  be  of  the  open/closed  loop  variety,  or  may 
be  rule  based,  or  a  combination  of  all  three.  The  control  system  adapter  chooses 
its  control  for  a  component  of  type  Eg'  from  the  set  of  feasible  controls  Ft.  The 
set  of  feasible  controls  Ft  then  serves  as  the  knowledge  base  for  the  Control 
System  Adapter.  The  inference  engine  chooses  the  member  of  the  knowledge 
base  that  meets  the  goal  chosen  by  the  goal  selector  as  close  as  possible.  The 
control  adapter  also  determines  if  reproduction  or  attrition  goals  can  be  met  and 
carries  out  this  directive. 
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FIGURE  3.4 


Control  System  Adapter 


The  Control  System  Adapter  for  the  air  combat  problem  determines  the 
aircraft  controls  that  causes  the  aircraft  to  "best"  follow  the  optimal  trajectory 
determined  by  the  Goal  Selector.  This  is  accomplished  by  a  rule-based  control 
system  dependant  upon  the  differential  game  model  chosen  in  the  System 
Identifier,  and  upon  the  performance  limits  of  the  aircraft. 

Lastly,  each  correlator  may  feed  back  information  to  another  correlator 
whenever  it  cannot  meet  the  performance  criteria  askea  for  by  the  other 
correlators. 
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3  4.  FRAME  BASED  SIMULATION  OF  SEMANTIC  SYSTEMS 


The  majority  of  large  scale  systems  that  are  under  study  in  today's 
technological  world  are  of  such  complexity  that  closed  form  analytical  tools 
provide  only  a  fraction  of  the  information  needed  to  understand  and  design  these 
systems.  The  paradigms  of  analysis  assume  a  homogeneous  form  of  a  system. 
Classes  of  these  paradigms  deal  with  systems  that  are  entirely  discrete 
dynamical  systems,  continuous  dynamical  systems,  decision  making  systems, 
linguistic  systems,  etc.  Real  world  systems  are  of  such  high  order  and  mixed 
type  that  simulation  techniques  are  a  must  to  understand  these  systems. 

Semantic  Systems  have  components  that  have  the  complexity  described 
above.  Another  order  of  complexity  is  added  in  the  self  modifying  nature  of 
Semantic  Systems.  Stability  and  controllability  of  certain  components  may  be 
desirable  sub-properties,  but  more  important  design  goals  might  be  reliability  and 
survivability  of  certain  groups  of  components.  This  is  indeed  the  case  for  air- 
combat  problems  in  which  the  designer  desires  survivability  and  reliability  only  on 
his  components.  Simulation  currently  is  the  only  method  for  design  and  analysis 
for  these  systems. 

How  do  we  go  about  modelling  and  simulating  semantic  systems  in  which 
the  existence  and  mix  of  objects  can  be  considered  part  of  the  state  of  the 
system?  The  paradigm  of  Frame  Base  Simulation  (FBSM)  is  proposed  as  a 
methodology  for  dealing  with  Semantic  Systems  and  other  highly  complex 
general  systems. 

Minsky  (1975)  proposed  the  idea  of  frames  and  scripts  to  represent 
knowledge  about  objects  and  events  typical  to  specific  situations  for  artificial 
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intelligence  (Al)  knowledge  representation.  This  is  a  method  of  organizing  the 
knowledge  representation  <n  a  way  that  directs  attention  and  facilitates  recall  and 
inference.  Minsky's  frames  were  based  on  "declarative"  knowledge.  Knowledge 
could  be  separated  into  a  general  set  of  procedures  for  manipulating  facts  of  all 
types,  and  a  set  of  facts  describing  a  particular  area  of  knowledge. 
Mathematically  this  is  akin  to  having  a  set  of  proof  procedures  that  are  applied  to 
a  specific  group  of  axioms.  Winograd  asserted  that  though  possible,  the 
restriction  of  frames  to  declarative  knowledge  is  overly  cumbersome.  Winograd 
added  the  idea  of  attached  procedures  to  Minsky's  frames.  These  attached 
procedures  represented  "procedural"  knowledge.  "Procedural"  knowledge  is 
knowledge  bound  to  the  use  of  the  knowledge,  or  programs  associated  with  how 
to  use  the  knowledge. 


Frames  from  the  Al  standpoint  provide  the  structure  that  allows 
interpretation  of  new  data  via  the  concepts  accumulated  by  previous  experience. 
This  format  for  Al  facilitates  "expectation-driven  processing",  looking  for  things 
based  on  the  conjectured  context.  An  example  of  a  generic  chair  frame  might 
be:26 


CHAIR  Frame 

Specialization-of: 

Number-Of-Legs: 

Style-Of-Back: 

Number-Of-Arms: 


Furniture 

an  integer  (Default  =4) 
straight,  cushioned, ... 
0, 1,  or  2 


Specialization-of,  Number-Of-Legs,  Style-Of-Back,  and  Number-Of-Arms 
are  "slots"  of  the  generic  chair  frame.  Slots  are  the  elementary  representational 


26 Barr  page  217 
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mechanism  of  "expectation-driven  processing".  Slots  provide  the  place  where 
knowledge  fits  within  the  larger  context  created  by  the  frame.  An  important 
property  of  frames  is  inheritance.  The  Specialization-of  slot  establishes  the 
"property  inheritance  hierarchy"  between  the  frames.  Descendant  frames  can 
inherit  information  of  parent  frames  via  this  mechanism.  A  particular  instantiation 
of  chair  inherits  the  same  slots  from  a  generic  chair  frame,  but  the  slots  are  more 
fully  specified: 


Roark's-CHAIR  Frame 
Specialization-of: 
Number-Of-Legs: 
Style-Of-Back: 
Number-Of-Arms: 


CHAIR 

127 

cushioned 

2 


We  want  to  model  a  semantic  system  in  which  the  semantic  state  is 
defined  by  (S,Rg,M).  We  start  with  the  set  E  =  {Eg':  i  e  1}  of  system  generation 
components.^8  The  set  E  for  the  air  combat  example  consists  of  aircraft,  tactical 
sites,  ground  defenses  (SAMS) ,  air  to  air  missiles,  ground  to  air  missiles,  air  to 
ground  missiles.  Each  Eg'  e  E  is  described  by  a  structural  model  Eg'1  and  a 
generic  FRAME. 


Frames  from  the  modeling  standpoint  provide  the  structure  in  which  the 
evolution  of  the  system  can  be  viewed.  This  is  "context-driven  modeling",  the 
component  is  modeled  based  on  the  context.  The  "slot"  again  serves  as  the 
elementary  representational  mechanism.  A  slot  for  FBSM  may  contain  in 


27|  sit  on  an  executive  office  chair  with  a  pedestal  base  due  to  my  many  hours 
sitting  in  front  of  my  computer 
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general,  subslots,  attached  procedures,  and  other  frames  as  is  the  case  for  Al. 

An  abridged  generic  aircraft  frame  would  be: 

Aircraft  Frame 

Mesarovic  Slot 

Parameters: 


State: 


Max  Clift: 

a  real 

Max  Thrust: 

a  real 

Max  G's: 

a  real 

Weight: 

a  real 

Wing  Surface  Area: 

a  real 

Oswald  Efficiency: 

a  real 

Wing  Aspect  Ratio: 

a  real 

X: 

a  real 

Y: 

a  real 

Z: 

a  real 

VLG: 

a  real 

aLG: 

a  real 

Heading  Rate: 

a  real 

Pitch  Rate: 

a  real 

Heading: 

[-Pi, Pi] 

Pitch: 

(-Pi/2,Pi/2) 

Fuel: 

a  real 

Number_Air_Air_Missiles:  [0,1 ,2, 3, 4, 5, 6] 

Is: 

Bank: 

(-Pi/2, Pi/2) 

Thrust: 

(0,  Max_Thrust) 

Clift: 

(0,  Max_Clift) 

if-needed  (procedure  to  update  state) 

Relational  State  Slot 

Tactic:  (LOS_Pursuit, 

LOS_Evade, 
LOS_Ground_Pursuit, 
LOS_G  rou  nd_E  vade 
Harrier_Attack__F  1 6 
FI  6_Evade_Harrier) 

Reproductive  Slot 

Air  To  Air  Missiles:  (Aim_9J,  Aim_9L) 

Missile  Description: 

Max  Off  Boresight:  [0,Pi] 

Min  Range:  a  real 

Max  Range:  a  real 

if-needed  (procedure  to  fire  missiles) 

Air  To  Ground  Missiles:  (****,  ****) 


Self  Attrition  Slot 

if-needed  (procedure  to  test  for  crash) 


2®Recall  that  each  member  Eg'  of  £is  a  Mesarovic  general  system. 
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A  generic  frame  is  defined  for  each  i  e  I.  Each  generic  frame  contains 
slots  for  modelling  the  component  i.  The  first  type  of  slot  defined  is  the 
"Mesarovic  Slot".  This  slot  contains  the  Mesarovic  structure  of  Eg’1,  a  structural 
model  of  Eg1.  The  declarative  portion  of  the  Mesarovic  slot  contains  subslots 
defining  the  global  state  set,  along  with  component  input  and  output  object  (Eg  j') 
sets  of  Eg  '.  A  global  state  is  defined  when  global  state  subslots  are  filled.  The 
procedural  portion  of  the  Mesarovic  slot  contains  attached  procedures 
representing  the  global  systems-response  function  for  Eg'.  These  attached 
procedures  indicate  how  to  determine  the  output  set  subslot  values,  and  state 
subslot  values  (for  the  dynamical  portion  of  the  system)  from  input  subslot 
values. 

The  state  object  slots  for  the  generic  aircraft  frame  consist  of  the  subslots, 
"Parameters"  x  "State" 

We  note  that  we  have  been  able  to  partition  the  global  state  set  into  a  constant 
portion  and  a  changing  portion.  The  "State"  portion  of  the  partition  corresponds 
to  the  segment  of  the  general  system  for  which  the  semi-group  property  holds. 
Mesarovic  has  shown  that  under  this  property  the  more  restrictive  notion  of  state 
space  can  be  defined  along  with  the  corresponding  transfer  function.  The  input 
object  set  slots  consist  of  the  "Control"  slots.  The  output  object  set  slots  are  not 
included  in  this  example  as  they  are  redundant  with  the  State  slots  for  an  aircraft 
model.  The  attached  procedure  representing  the  aircraft  transfer  function 
consists  of  the  discrete  time  version  of  the  aerodynamic  equations  of  motion. 

The  second  type  of  slot  defined  for  each  generic  frame  is  the  "Relational 
State  Slot".  This  slot  contains  frames  indicating  the  feasible  relation  r  e  Ft 
currently  representing  the  relational  state  of  the  component  i.  This  is  a  example 
of  when  a  frame  slot  may  contain  another  frame.  A  relational  frame  is  defined  for 
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every  re  fl1.  The  relational  frames  will  be  described  later  but  an  example  for  the 
air  combat  problem  is  shown  below. 


Harrier  Attack  of  FI  6  Frame 
Harrier  Description: 
Speed: 

Kill  Radii: 

Kill  Eccentricity: 
FI  6  Type  Target: 

F16  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 
Max  Turn  Radius: 


a  real 
a  real 
a  real 

an  aircraft  frame 

a  real 
a  real 
a  real 
a  real 


if-needed  (procedure  to  determine  Harrier  control) 


The  next  slot  of  interest  is  the  "Reproductive  Slot".  This  slot  contains 
subslots  defining  the  type  of  components  that  may  be  reproduced  (EsJ),  the  initial 
state  object  and  relation  (Rj)  for  a  reproduced  component,  and  conditions  under 
which  reproduction  may  take  place  (CrJ).  The  declarative  portion  of  the  slot 
contains  a  subslot  whose  value  is  the  frame  type  of  the  object  to  be  reproduced. 
The  condition  for  reproduction  and  initial  relation  are  determined  by  both 
declarative  and  procedural  subslots.  The  parameterization  of  the  reproductive 
conditions  (CpJ)  may  be  specified  by  declarative  slots,  while  the  procedure  for 
testing  for  (CrJ)  is  done  via  an  attached  procedure. 

The  reproductive  slot  for  the  generic  aircraft  frame  contains  the  type  of 
missiles  that  may  be  fired  (reproduced).  The  type  of  air  to  air  missile  subslot 
includes  a  frame  subslot  containing  a  specialization  of  an  air  to  air  missile  frame 
(Aim_9L,  Aim_9J).  There  are  also  declarative  subslots  describing  the 
capabilities  of  a  missile.  These  conditions  are  tested  by  the  attached  procedure 
to  determine  if  a  missile  should  be  fired. 


Slots  are  also  defined  for  attrition  of  both  types.  The  self  attrition  slot 
contains  an  attached  procedure  to  determine  if  the  conditions  C3A  have  been 
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met.  These  conditions  for  the  aircraft  frame  include  an  attached  procedure  for 
testing  for  out  of  fuel,  flying  into  the  ground,  or  exceeding  the  design  of  the  air 
frame. 


The  causative  attrition  slot  contains  attached  procedures  for  determining  if 
the  conditions  are  met  for  removing  a  frame  instantiation  of  type  j.  This 
type  of  slot  is  not  present  for  the  aircraft  frame  assuming  no  guns. 

We  now  turn  our  attention  to  the  relational  frames.  A  generic  relational 
frame  is  specified  for  each  rs  fl*.  The  declarative  portion  of  the  frame  carries 
the  object  state  set  for  the  relation.  This  set  carries  the  paramterization  of  how 
the  relation  is  to  be  fitted  to  the  component  frame  to  which  it  is  attached.  The 
declarative  portion  also  contains  slots  containing  frames  that  contain  instantiation 
of  the  components  that  serve  as  the  domain  of  the  relation.  The  procedural 
portion  contains  an  attached  procedure  to  determine  the  inputs  of  component  i 
based  on  the  declarative  portion  cf  the  slot  (domain  components  and  relational 
state).  An  instantiation  of  a  relational  frame  occurs  when  a  component  i  chooses 
relation  r  e  Ft. 

The  "Harrier  AttacK  of  FI  6"  frame  contains  a  slot  for  a  specific 
instantiation  of  an  FI  6  frame  indicating  the  current  target  for  the  Harrier 
instantiation.  The  description  slots  describe  a  structural  model  for  both  the  FI  6 
and  Harrier  instantiations.  These  models  are  of  much  lower  order  than  the  actual 
structural  models  specified  in  the  Mesarovic  slot  of  the  component  frame.  These 
lower  order  models  are  used  so  that  an  actual  solution  to  the  relation  can  be 
computed. 

The  structure  and  management  of  frame  instantiation  is  what  separates 
FBSM  from  simply  being  a  data  structure.  Inheritance  allows  one  frame  to  be 
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related  to  another.  The  generic  aircraft  frame  was  defined  above.  We  may 
further  specify  aircraft  types  by  the  use  of  inheritance.  An  FI 6  type  aircraft  would 
have  the  same  slots  as  the  generic  aircraft  frame  but  the  Parameter  slots  would 
be  specified.  Similarly  to  the  Al  case  an  Ft  6  would  be  a  specialization  of  the 
aircraft  frame.  The  same  applies  to  missiles  where  we  have  used  the 
specialization  of  AIM_9L  and  AIM_9J.  A  specialization  of  a  frame  is  different 
than  an  instantiation.  A  instantiation  of  a  frame  implies  the  existence  of  a  system 
component.  A  FI 6  is  a  type  of  aircraft  but  Blue  Fighter  1  which  is  an  instantiation 
of  a  FI  6  is  an  actual  component.  Reproduction  is  the  act  of  instantiation  of  a 
specialization  of  a  frame.  The  case  where  Blue  Fighter  1  (F16)  fires  an  AIM_9L 
missile  means  that  the  Blue  Fighter  1  will  cause  the  instantiation  of  a  missile  that 
inherits  the  properties  of  an  AIM_9L  missile  frame. 

Frames  from  an  Al  standpoint,  by  supplying  a  place  for  knowledge,  create 
the  possibility  of  missing  or  incompletely  specified  knowledge.  The  slot 
mechanism  permits  reasoning  based  on  seeking  confirmation  of  expectations 
(I.E..  fill  in  the  blanks).  The  use  of  attached  procedures  can  be  used  to 
determine  the  values.  For  modelling,  initially  we  assume  all  slots  are  filled. 
Subsequent  slot  values  and  instantiations  are  determined  with  the  use  of 
attached  procedures  and  allow  for  FRAME  interdependencies.  An  FBSM 
executive  is  responsible  for  the  management  of  the  simulation  system.  This 
executive  keeps  track  of  instantiations,  removes  instantiations,  and  manages 
iteration  events  for  slot  values.  The  FBSM  of  the  air  combat  environment  has 
been  implemented  in  PASCAL  and  currently  runs  on  a  SUN  4. 
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4.  AIR  COMBAT  ENVIRONMENT  SIMULATION 

The  air  combat  environment  simulator  will  now  be  described.  The 
simulator  will  be  described  in  terms  of  Semantic  Systems  and  Frame  Based 
Simulation  (FBSM)  described  in  Section  (3). 

The  index  set  I  of  system  generation  component  set  E  consists  of  Aircraft, 
Ground  Targets,  and  Ground  Defenses.  We  note  that  in  our  stage  one  simulator 
we  are  not  including  air  to  air  missiles,  air  to  ground  missiles,  and  ground  to  air 
missiles.  One  additional  advantage  to  FBSM  is  that  a  model  of  the  causative 
attrition  rules  of  a  component  may  be  absorbed  into  the  Frame  model  of  the 
components  that  can  reproduce  the  component. 

The  section  will  be  organized  in  terms  of  the  component  types  of  the  index 
set  I.  Each  component  type  will  have  a  section  describing  the  structural  model 
E’s'  for  Es'  e  E,  followed  by  Frame  description.  The  first  component  we  will 
consider  will  be  aircraft. 

4.1.  AIRCRAFT 

This  section  of  the  simulator  description  will  be  devoted  to  the  aircraft 
components.  We  first  will  discuss  the  aerodynamic  model  for  an  aircraft,  a 
continuous  time  dynamical  model.  The  discrete  time  numerical  model  for  the 
simulation  will  then  follow.  The  discussion  will  then  continue  with  a  probabilistic 
model  for  the  missile  capabilities  of  the  aircraft.  Lastly  we  will  define  the  aircraft 
component  frame. 
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4.1.1.  AERODYNAMIC  MODEL 

There  are  four  aerodynamic  forces  that  must  be  considered,  gravity, 
thrust,  lift,  and  drag.  The  aircraft  frame  has  three  axis:  longitudinal,  lateral,  and 
vertical.  The  longitudinal  axis  is  through  the  aircraft  body.  The  lateral  axis  is 
through  the  aircraft  wings  perpendicular  to  the  longitudinal  axis.  The  vertical  axis 
is  perpendicular  to  the  plane  formed  by  the  longitudinal  and  lateral  axis. 

The  following  angles  define  the  aircraft  movement  in  three  dimensional 
euclidian  coordinates:  0,  <j>,  and  x.  The  heading  of  the  aircraft  in  the  x-y  plane  is 
defined  by  0.  This  angle  is  measured  counterclockwise  from  the  x-axis. 

The  rotation  of  the  aircraft  about  its  lateral  axis  is  defined  by  the  pitch 
angle  <t>.  The  pitch  angle  is  measured  clockwise  from  the  horizontal  x-y  plane. 
Pitch  angle  defines  if  the  aircraft  is  flying  nose  up  or  down  with  respect  to  the 
horizontal  plane. 

The  last  angle  x  defines  the  rotation  of  the  aircraft  about  its  longitudinal 
axis.  The  bank  angle  x  is  measured  clockwise  from  the  x-y  plane.  Bank  angle 
defines  the  aspect  of  the  wings  with  respect  to  the  x-y  plane. 

Thrust  is  the  force  applied  by  the  aircraft  engine  along  the  forward 
longitudinal  axis  of  the  aircraft.  If  T  is  the  magnitude  of  the  thrust  (NT),  then  the 
components  are, 

Tx  =  Tcospcos0 
(4.1.1)  Ty  =  Tcospsin© 

Tz  =  -Tsinb 
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The  lift  force  is  directed  along  the  vertical  axis  of  the  aircraft.  Lift  is  the 
force  induced  by  the  wings  of  the  aircraft.  If  L  is  the  magnitude  of  the  lift  its 
components  are: 

Lx  =  L(sin<J>cos0cosx  +  sinGsinx) 

(4.1 .2)  LY  =  L(sin<j>sinGcosx  -  cosGsinx) 

\-2  =  Lcosxcos4> 

The  magnitude  of  the  lift  force  L  can  be  related  to  the  unit-less  lift  coefficient  C|_ 
by. 

(4.1.3)  L  =  oClvlg2S/2 
where 

w'LG  s  Aircraft  Longitudinal  Axis  Velocity 
a  a  Air  Density  (kg/m3) 

S  =  Aircraft  Wing  Area  (m2) 

Drag  is  the  force  along  the  longitudinal  axis  opposite  the  direction  of  travel. 
The  drag  force  arises  from  resistance  of  the  aircraft  movement  through  the  air.  If 
DG  is  the  magnitude  of  the  drag,  its  components  are: 

DGx  =  DGcospcosG 

(4.1.4)  DGy  =  DGcos<|>sin0 
DGz  =  -DGsinp 

The  magnitude  of  the  drag  force  D  can  be  split  into  two  components,  the 
parasite  drag  Cqq  and  the  induced  drag  Cpj.  The  parasite  drag  Cqq  is  due  to 
the  profile  drag  of  the  airfoil,  skin  friction  drag,  pressure  drag,  and  interference 
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drag  of  the  other  aircraft  components.  The  induced  drag  is  related  by  Prandtl’s 
lifting  line  theory  to  the  lift  coefficient  C|_  by 

(4.1.5)  CDi  =  CL2/(near) 
where 

ar  s  Aircraft  Wing  Aspect  Ratio 
e  s  Oswald  Wing  Efficiency  Factor 

The  magnitude  of  the  drag  D  is  then  related  to  these  coefficients  by, 

(4.1 .6)  DG  =  ovLG2S(CD0  +  CL2/near)/2 

The  equations  of  motion  can  be  found  by  equating  the  forces  ((4.1.4), 
(4.1.1),  (4.1.2))  for  each  of  the  x,  y,  z,  components  along  with  (4.1.3)  and  (4.1.6). 
This  gives 

^LG*  =  9((t/w-(B  +  CLC)vLG2)  +  sin<j>) 

<l>*  =  g(C|_DvLGcosx  -  cos<{>/vLG) 

(4.1 .7)  6*  =  -  gC|_D  vj_Gsinx/cos<J) 
x*  =  vLGcos<}>cose 

y*  =  VLGcos0sin6 
z*  =  -  VLGsin<}> 

where 

B  =oCDOS/2W 
C  =aS/2Wnear 
D  =  oS/2W 

W  s  Aircraft  Weight  (NT) 

V|_q  =  Aircraft  Longitudinal  Axis  Velocity 
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The  trajectory  of  the  aircraft  is  determined  by  three  controls;  T  (thrust),  t 
(bank  angle),  and  C|_  (coefficient  of  lift).  All  three  controls  are  assumed  to  be 
instantaneously  adjustable  between  the  minimum  and  maximum  values. 

Aircraft  dynamics  are  limited  by  a  number  of  factors.  Thrust  (T)  is  limited  to 
a  maximum  amount  available  (Thrust__Max).  Maximum  thrust  is  a  characteristic 
of  the  design  performance  of  the  aircraft  engine.  We  are  also  assuming  fuei 
limitations.  Once  fuel  is  exhausted  thrust  is  obviously  no  longer  available.  The 
fuel  burn  equation  may  be  written  as  follows: 

(4.1.8)  F*  =  -  efT 

where 

F  =  fuel  amount 
ef  s  engine  efficiency 
T  s  time  elapsed 

A  maximum  amount  of  lift  is  also  available  at  any  velocity.  The  maximum 
lift  available  is  related  to  the  maximum  value  of  C|_  (C|__Max).  The  maximum 
coefficient  of  lift  (Ci__Max)  is  a  parameter  of  design  performance  of  the  aircraft 
airframe.  The  maximum  coefficient  of  lift  (C|__Max)  determines  the  stall  velocity 
of  the  aircraft.  An  uncontrollable  loss  of  altitude  occurs  when  an  airplane  is  in  a 
stall.  The  stall  velocity  is  defined  as  the  velocity  (v^q)  below  the  point  at  which 
the  pitch  rate  ($•)  defined  in  (4.1 .7)  can  be  held  to  zero,  and  is  given  by, 

r  COS<()  -|1/2 

(4.1.9)  ^LG-Stall  =  — - 

L  C[_Dcosx  J 

The  maximum  load  factor  of  the  airframe  is  another  design  parameter. 
The  maximum  load  factor  is  given  in  G's  which  is  equal  to  the  force  of  gravity  on 
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an  object  at  rest  at  sea  level.  The  instantaneous  number  of  G's  for  an  aircraft  is 
given  by: 

Vi  p  r  -.1^2 

G  =—  (6#cos4>)2  +  (4>*)2 

g  L  J 

where  g  is  the  gravitational  acceleration  constant. 

4.1.2.  NUMERICAL  METHOD 

We  will  now  describe  the  numerical  method  for  simulating  the 
aerodynamic  model  discussed  in  Section  4.1.1.  The  first  two  equations  of 
(4.1 .54)  can  be  decoupled  from  the  rest  and  are  of  the  vector  form, 

(4.1.10)  V#  =  F(V,U) 

where  F(«,«)  is  a  C^1)  function  from  Rn+m  to  fl0,  V  is  a  n-dimensional  state 
vector,  and  U  is  a  m-dimensional  control  vector. 

The  following  notation  will  be  needed.  We  define  tj  =  tj_i  +  h,  Vj  =  V(tj), 
V*  =  V*(tj),  and  Uj  =  U(tj).  Denoting  Fy(*,»)  and  Fy(*,»)  as  the  matrix  of  partials 
with  respect  to  the  V  and  U  coordinates  of  F(»,»)  respectively  we  define 
Fv'  =  Fy(Vj.Uj)  and  Fy'  =  Fy(Vj.Uj).  We  can  now  define  the  first  order 
approximation  to  (4.1 .10)  at  time  tj+i  around  tj  as, 

(4.1.11)  Vi+1  =  Vj  +  Fvi(Vj+1  -  Vj)  +  Fyi(Ui+1  -  Uj) 

where  we  have  used  the  fact  that  Vj  =  F(Vj,Uj).  The  trapezoidal  rule  allows  us  to 
approximate  Vj+1  in  terms  of  Vj  and  is  given  by: 


(4.1.12)  Vl+1  =  Vj  +  h(Vi+i  +  V*j)/2. 
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Substituting  (4.1.12)  into  (4.1.11)  we  get, 

(4.1.13)  Vi+i  =  Vj  +  Fv'Vj  + 

hFv'Vi+1 12  *  hFv'V‘i/2  +  fu'(Ui+1  -  Uj). 


Collecting  terms  we  have, 

(4.1.14)  (I  -  hFv'/2)Vi+1  =  (I  +  hFv'/2)V*j 

+  FvV,  +  Fui(Ui+1  -  Uj). 

where  I  is  the  nxn  identity  matrix.  Multiplying  (4.1.14)  by  (I  -  hFyi/2)"1  we  have, 

(4.1.15)  Vi+1  =  (I  -  hFvi/2)-1  (t  +  hFv'/2)Vj 

+  (I  -  hFvi/2)'1(FviVi  +  Fu'(Ui+1  -  Uj)), 


Equation  (4.1.15)  and  equation  (4.1.12)  form  the  difference  equation 
model  for  vLG*  and  of  (4.1.7),  the  aircraftdynamics  differential  model.  We 
now  give  Vj,  Uj,  Fv',  and  Fv'  for  the  aircraft  structural  model.29 

vi  =  t  vLGi  ^  U  =  [  Tj  Cy  Xj  ]T 


(4.1.16) 


-2g(B  +  CLj)vLGi 
gCyDcosxj  -  gcos<t>j/(vLGi)2 


gcos<|)j 

gsin0j/vLGi 


■gC(vLGi)2 

gDvLGicosxj 


0 

-gCyOvLGiSinxj 


29[  ]T  denotes  the  transpose  of  the  vector  [  ] 
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We  now  turn  our  attention  to  the  differential  equation  for  0*  from  (4.1.7).  A 
close  examination  indicates  that, 

(4.1.17)  0*i+l=0*(ti+1)  =  -gCL(i+l)D^LG(i+1)sinxi+1/cos<j)j+1 

is  determined  by  Vj+-j  and  U-,+i  defined  in  (4.1.16)  where  V-l+1  is  determined  by 
the  methodology  defined  in  (4.1.15)  and  (4.1.12)  The  trapezoidal  rule  is  once 
again  used  along  with  (4.1.17)  so  that, 

(4.1.18)  0i+1  =  0(tj+1)  =  0i+1  +  h(0*i+1  +  0*,)/2 

The  last  equations  for  x*.  y*.  and  z*  of  (4.1.7)  are  decoupled  and  can  be 
determined  in  terms  of  Vj+1 ,  Uj+-)  and  0j+-j  and  are  given  by 

X*j+i  =  V|_G(i+1  )COS<|)j+-jCOS0j+i 

(4.1.19)  y*  +1  -  l'LG(i+ 1  )cos<f>i+i  sin0j+  -j 
z*i+1  =  '^LGCi+l  )sin<^i+1 

We  use  the  trapezoidal  rule  one  final  time  along  with  (4.1.19)  to  get: 

xi+i  =  xj  +  h(x*i+1  +  x*j)/2 

(4.1.20)  yi+1  =yi  +  h(y*i+i  +y*)/2 
zj+i  =  Zj  +  h(z*i+1  +  z*j)/2 
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4.1.3  PROBABILISTIC  MISSILE  MODEL  DEVELOPMENT 

We  will  now  derive  the  probabilistic  model  for  missile  firing  envelopes. 
This  model  will  serve  as  the  causative  attrition  rules  Gj  between  aircraft.  The  true 
nature  of  these  models  are  classified  and  we  only  seek  a  plausible 
approximation.  The  actual  details  of  the  model  is  not  critical,  but  the 
methodology  to  determine  the  firing  and  risk  surfaces  is  critical  to  our 
methodology. 

We  start  by  asymptotically  looking  at  the  deterministic  case  for  a  constant 
speed  missile  with  bounded  turn  rate  against  a  straight  flying  aircraft  with 
constant  speed  taking  place  at  fixed  altititude.  Justification  for  this  asymptotic 
case  is  that  the  vast  majority  of  pilots  shot  down  never  realized  they  had  been 
fired  on  and  hence  took  no  evasive  maneuvers.  The  missile  is  assumed  to  have 
a  finite  life  time  which  is  the  burn  time  of  it  motor.  Our  problem  is  then  to 
determine  the  maximum  range  from  the  target  aircraft  that  the  missile  can  reach. 
The  coordinate  frame  we  will  work  in  is  centered  at  the  target  aircraft  with  the 
aircraft’s  velocity  vector  aligned  with  the  y-axis.  This  coordinate  system  is  shown 
in  Figure  4.1.1. 


Target  Aircraft 


X 


COORDINATE  SYSTEM  RELATIVE  TO  TARGET  AIRCRAFT 


FIGURE  4.1.1 

The  analysis  is  started  by  assuming  the  missile  is  infinitely  maneuverable. 
This  corresponds  to  the  maximum  firing  range  from  any  point  in  space  since  the 
missile  can  always  be  fired  toward  the  optimal  intercept  point  regardless  of  the 
position  and  direction  of  the  aircraft  firing  the  missile. 


The  equations  of  motion  are: 


(4.1.21) 


X*  =  VMsin<D 
Y*  =  -VA  +  VjyjCosO 


where 

VVj  =  velocity  of  missile 
Va  =  velocity  of  target  aircraft 
O  =  missile  heading  control. 


The  Hamiltonian  of  the  optimal  minimum  intercept  time  control  problem  is: 


(4.1.22) 


H  =  1  +  rxVjvjSinO  +  rY^cos<D  - 
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The  adjoint  equations  are  then: 

r  y*  =  0 

(4.1.23) 

.  rv  =  0 

The  optimai  control  found  by  minimizing  the  Hamiltonian  with  respect  to  <t> 
is,  O  =  arctan^y/Tx)  +  it.  The  fact  that  the  adjoints  are  constant  implies  that  <1> 
is  constant  and  may  be  found  geometrically  by  solving  for: 

X(tf)  =  X(t0)  +  tf  Vj^sinO  =  0 

(4.1.24) 

Y(tf)  =  Y(t0)  -  tfVA  +  tf  l^Mcos<D  =  0. 

The  heading  angle  <X>  is  then, 

(4.1 .25)  <X>  =  arctan(X(t0),Y(t0)  -  tf  VA)  +  it 

Our  interest  lies  in  determining  the  manifold  of  the  maximum  range  of  the  missile 
for  tf  =  missile  bjrn  time.  This  leads  to  the  equation: 

(4.1 .26)  X2  +  (Y- ^aTbt)2  =  ( VMTBT)2. 

The  maximum  range  for  a  missile  fired  optimally  at  the  intercept  point  is 
thus  an  eccentric  circle  in  front  of  the  target  aircraft  by  the  distance  the  target  can 
fly  during  the  life  of  the  missile  with  radius  of  the  distance  the  missile  can  travel 
during  its  life.  An  example  is  shown  in  Figure  4.1 .2  below. 
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flaxi  inuiTi  Range  Singular  Surface 


FIGURE  4.1.2 


We  now  may  consider  the  case  when  the  missile  has  a  maximum  turn 
rate.  The  case  aoove  serves  as  a  singular  surface  for  when  a  missile  is  perfectly 
aligned  for  intercept  when  fired.  If  the  missile  is  not  aligned  then  the  maximum 
range  will  be  reduced  as  the  missile  wastes  time  and  energy  turning  to  an 
intercept  course. 


The  equations  of  motion  are  now: 
X*  =  V^sinO 

(4-1-27> 


where 

1 =  velocity  of  missile 

Vpn  =  velocity  of  target  aircraft 

<X>  =  missile  heading  relative  to  aircraft  heading 

0  =  missile  turn  control  (-1  £  <{>  <  1 ) 

R  =  minimum  turn  radius  of  missile. 
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It  should  be  noted  that  <t>  is  no  longer  a  control,  but  now  a  state  variable.  The 
Hamiltonian  of  the  optimal  minimum  intercept  time  control  problem  is: 

(4.1 .28)  H  =  1  +  Tx  VMsin<D  +  rY  VMcos<D  -  TY  VA  +  r<t> 

The  adjoint  equations  become: 

ry  =  o 

(4.1.29)  Iy*  =  0 

V  =  - VmOxCOsO -  rV VM^) 

The  optimal  control  <j>  is  found  by  minimizing  the  Hamiltonian  with  respect  to  4> 
yielding: 

(4.1.30)  4>  =  -signfr^) 

The  terminal  conditions  are  : 

(4.1.31)  X(tf)  =  Y(tf)  *  0;  <D(tf)  =  n  +  030. 

Transversal ity  also  yields  r<j>(tf)  =  0.  The  terminal  conditions  for  the  other 
adjoints  follow  from  the  terminal  condition  that  min^  H  =  0  yielding, 

sin0  cose 

(4.1.32)  rx=  -  rY - . 

+  v^cose  +  vAcose 


30e  can  be  viewed  as  the  angle  at  which  intercept  is  approached  infinitesimally 
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This  lead?  *  ^ 


(4.1.33)  lV(tf)  = 


^M+  Kacos6 


(sinecosO(tf)  -  cos9sin<t>(tf)) 


-Vi 


M 


VM+VAcosQ 


sin(0  -  <D(tf)) 


=  0. 


This  implies  that  <J>*(tf)  =  0.  The  conclusion  is  that  all  trajectories  terminate  on  a 
singular  surface  on  which  the  missile  is  flying  a  straight  intercept  course.  The 
missile  thus  turns  maximally  until  it  reaches  a  point  that  O(t)  matches  the  angle 
for  X(t)  and  Y(t)  that  would  be  optimal  for  the  infinitely  maneuverable  missile.  The 
missile  then  switches  to  a  straight  lead  intercept  trajectory.  A  typical  trajectory 
shape  is  shown  in  Figure  4.1.3  below. 
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FIGURE  4.1 .3 
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Letting  t  be  retro  time,  and  x-|  be  the  retro  time  at  which  the  trajectory  enters  the 
singular  surface  we  have  the  equations  of  motion  for  x  <  x-| , 

O(x)  =  7i  +  0 

(4.1.34)  X(x)  =xV'Msin0 

Y(x)  =  x(VA  +  VMCOS0) 

The  equations  for  motion  for  x  >  x-j  when  <j>  =  1  are: 

O(X)  =  7C  +  0  -  -  (X  —  Xi ) 

R 


(4.1.35) 


X(x)  =  Rcos 


+  X(x-| )  -  Rcos0 


Y(x)  =  -Rsin 


(x-xi ) 


+  V^(x-xi )  +  Y(x-|)  +  Rsin0 


Our  interest  lies  in  how  the  maximum  range  envelope  varies  with  deviation 
from  the  optimal  missile  intercept  heading  for  the  infinitely  maneuverable  case. 
Numerically  we  can  solve  for  the  maximum  range  as  a  function  of  heading  error 
from  the  Maximum  Singular  Surface  Range  for  the  infinitely  maneuverable 
missile  solved  earlier.  The  outermost  range  envelope  corresponds  to  the  case 
where  the  missile  heading  exactly  matches  the  firing  heading  if  the  missile  was 
infinitely  maneuverable.  Thus  for  each  X  and  Y  there  corresponds  the  optimum 
firing  angle  <E>*(X,Y)  which  is  the  solution  to  (4.1.25).  Then  for  each  angle  ft  = 
arctan(Y.X)  we  can  determine  numerically  the  maximum  X,  Y  pair  for  a  fixed  error 
offset  from  <I>*(Ytanft,Y).  These  plots  are  given  in  Figure  4.1 .4. 
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FIGURE  4.1.4 
FIRING  RANGES 


The  next  asymptotic  case  will  now  be  considered.  We  consider  an 
infinitely  maneuverable  aircraft  against  a  missile  with  bounded  turn  radius.  Once 
again  we  assume  the  encounter  takes  place  at  fixed  altitude  with  constant 
velocity  for  both  vehicles.  Since  the  aircraft  is  now  infinitely  maneuverable  we 
assume  the  aircraft  always  aligns  its  velocity  vector  outward  along  the  line  of 
sight  between  the  aircraft  and  missile.  The  coordinate  frame  we  will  now  work  in 
is  centered  at  the  missile  with  the  Y-axis  aligned  with  the  missiles  velocity  vector. 
The  coordinate  system  is  shown  in  Figure  4.1.5. 


MISSILE 


COORDINATE  SYSTEM  RELATIVE  TO  MISSILE 


FIGURE  4.1.5 


The  equations  of  motion  written  in  polar  coordinates  are: 


(4.1.36) 


r*=  VA~  \/mcosA 


r  <b  sinA  -i 

f-Vfo  —  - 

L  R  r  J 


where 


r  =  distance  of  aircraft  from  missile 

A  =  heading  of  aircraft  relative  to  missile  heading 

Vm  =  velocity  of  missile 

VA  =  velocity  of  target  aircraft 

()>  =  missile  turn  control  (-1  <  <f>  s,  1) 

R  *  minimum  turn  radius  of  missile. 


The  Hamiltonian  is: 


r  VMY  i  V^sinA 

(4.1.37)  h  .  1  +  vArr  +  rA  —  +  rA - rrvMcosA 

L  R  J  r 
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The  adjoint  equations  are  then 

V^sinA 


(4.1.38) 


rr*  =  -rA- 

r- 

rA*=  -VM[rA— —  *  TfSinA 


Minimizing  H  with  respect  to  4>  implies  the  optimal  control: 

(4.1 .39)  <|>  =  -sign[TA] 

The  terminal  conditions  require  some  explanation.  It  should  be  obvious 
that  the  only  way  r  =  0  can  be  reached  is  along  the  singular  surface  for  which  A  = 
0.  Thus  we  seek  those  trajectories  that  reach  the  singular  surface  A=0,  r  =  s  in 
minimum  time.  Symmetry  arguments  clearly  indicate  that  <|>  =  1  in  the  right  half 
plane  and  <(>  =  *1  in  the  left  half  plane.  Assuming  the  missile  has  a  reasonably 
long  burn  time,  we  can  use  for  the  boundary  of  the  range  of  the  missile: 

•  $ 

(4.1.40)  A  =  VM - 

R 

Letting  x  once  again  be  retrograde  time,  and  x-j  be  the  retrograde  time 
when  the  breakaway  from  A  =  0  occurs,  we  have  for  the  right  half  plane  (<>  =  1 ) 
for  x  >  x-j : 

A(x)  =  —  (x-  x^ 

R 

(4.1.41) 

r(x)  =  Rsin  F  —  (x-x1 )  1  +  VA(x  -  t-j )  +  x-,  ( VM  -  VA) 

L  R  J 


To  find  the  maximum  capture  range  envelope  we  first  note  that  we  can  relate  rto 
A  by, 

(4.1 .42)  r  -  RsinA  +  ( -  V^jT qj  -  R»A 
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where  Tgy  is  the  life  time  of  the  missile  engine  burn.  The  two  figures  below 
show  how  the  maximum  range  envelope  varies  with  missile  burn  time  and 
minimum  turn  radius.  The  first  figure  shows  how  the  maximum  range  envelope 
varies  for  fixed  minimum  turn  radius  R,  while  Tgy  is  varied.  The  second  figure 
indicates  how  the  maximum  range  envelope  varies  as  R  is  varied  while  Tgy  is 

held  fixed. 


Range  As  A  Function  Of  Burn  Time 


FIGURE  4.1.6 
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Range  As  A  Function  Of  Turn  Rate 


We  now  will  splice  our  deterministic  asymptotic  results  together  into  a 
probabilistic  missile  effectiveness  model.  Arguably  the  results  are  not  rigorous. 
We  seek  only  a  plausible  model  of  sufficient  complexity  that  our  methodology  can 
be  exercised  in  more  than  a  trivial  fashion. 


Earlier  we  mentioned  that  the  vast  majority  of  pilots  shot  down  never 
realized  they  had  been  fired  upon.  Therefore  we  define  Py  =  probability  that  a 
missile  launch  is  undetected  by  the  fired  upon  aircraft.  We  can  thus  use  the 
deterministic  model  summarized  by  equation  (4.1.26).  Rewriting  equation 
(4.1 .26)  in  line  of  sight  coordinates: 

(4.1.43)  r  =  V,aTBtCOS(AAsp)  + 


(^BT)2  ~  ( ^ATBTs'n(AASP))2 
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where  AABp  is  the  aspect  angle  which  is  defined  as  the  angle  measured 
clockwise  between  the  aircraft  heading  and  the  line  of  sight,  and  r  is  the  line  of 
sight  distance. 

The  next  step  is  to  modify  the  model  given  by  equation  4.1.43  from  a 
deterministic  model  to  a  probabilistic  one.  This  modification  will  account  for  the 
fact  that  all  of  the  parameters  TBT,  yA,  etc.  vary  probabilistically.  The 
probability  that  the  missile  will  destroy  an  aircraft  from  a  line  of  sight  distance  r, 
where  the  aircraft  has  aspect  angle  AABp,  is  given  by: 

r  pexp(-ad),  if  d  >  0 

(4.1.44)  P(K|  r,AASP,U)  = 

i-  p,  otherwise 

where 

d  =  r-  VATB-|-cos(AASp) - 

r  o  o-i  1/2 

(^MTBT)2-(^ATBTsln(AASP))2  J 


The  non-negative  a  models  the  variability  around  the  nominal  parameters  yA, 
t/f^,  and  TBj.  Last  ditch  end-game  maneuvers,  electronic  counter  measures, 
terrain  masking,  etc  are  modeled  by  the  parameter  0  <  p  <  1 . 

Equation  (4.1.26)  was  the  result  when  the  missile  was  assumed  to  be 
infinitely  maneuverable  so  that  missile  off-boresight  angle  was  not  considered. 
The  off-boresight  angle  firing  error  from  equation  (4.1 .25)  can  be  written  as: 

(4.1 .45)  <De  =  AAgp  -  Aqb  +  *  + 

arctan(rsin(AASP),rcos(AASP)  -  TBTyA) 
where  <De  is  the  off-boresight  angle  error  and  AqB  is  the  off-boresight  angle 
measured  clockwise  between  the  missile  heading  and  the  line  of  sight.  The  line 
of  sight  coordinate  system  is  shown  in  Figure  (4.1 .8). 
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Missile 


Aircraft 


Line  Of  Sight  Coordinates 
FIGURE  4.1.8 

The  effect  of  Agse  was  shown  in  Figure  4.1.4.  As  it  turns  out,  the 
relationship  between  the  maximum  missile  range  and  Aqb©  is  highly  nonlinear 
and  a  closed  form  solution  does  not  exist.  The  radii  of  the  eccentric  circles  are 
found  from  Equations  (4.1.35)  to  be: 

(4.1.46)  p2  =  X2  +  (  y-vA)2 

=  ^MtBT  ~  ^m(tbtxi  )] 

+  2RM2[l-cos  [  ~  (TBrxl)ll 
L  L  rm 

+  2t1VMRsin[  ^  (TBr-c1 )  1 

L  rm  j 

where  the  parameters  are  those  of  Equations  (4.1 ,27).21  Define  8'  by 

(4.1.47)  8'  =  arctan(X(t0),Y(to)  -  tf  VA), 


31TBj  is  still  the  missile  burn  time. 
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which  is  the  angle  between  the  initial  position  of  the  missile  and  the  heading  of 
the  aircraft.  <De  can  then  be  expressed  as: 


(4.1.48) 


<De  =  e  -  e*  - 


(TBT-Xl)- 


The  parameter  C  is  defined  below,  strictly  for  convenience  as: 


(4.1.49)  C  = - (TBT-*l) 

rM 


The  problem  is  to  determine  the  unknowns  0  and  from  <J>e  so  that  p  may  be 
calculated.  Again,  no  closed  form  solution  exists.  Our  methodology  is  to  use  a 
second  order  Taylor  expansion. 


Assuming  <De  is  negative,  the  first  step  is  the  equation  found  by  combining 
second  order  Taylor  expansions  of  equations  (4.1 .35)  with  (4.1 .47): 

X  2VMTBTSin9-RM<C)2cose 

(4.1.50)  tan0'  =  -  «  - - - 

Y- ^At BT  2  W\ QjcosQ  -  RM(C)2sin0 

.  fo  ♦  -i  r  rm(C)2  ii 

=  tan  0  -  tan  1  - 

L  L  2^mTBT  JJ 

This  leads  us  to  conclude  that: 

i  r  RM(C)2  i 

(4.1.51)  0’«  0-tan-1  - 

*•  2V/mtbt  • 


89 


A  second  order  Taylor  expansion  can  now  be  applied  to  equation  (4.1 .51), 
yielding 


-e'  ♦  **r  —  i 

2  *-  ^MtBT 


The  results  of  equations  (4.1.51)  and  (4.1.48),  valid  for  negative  <De,  can  be 
extended  to  determine  the  radius  of  the  eccentric  circle  missile  range  p 
approximately,  as  a  function  of  |  <De  | ,  from  equation  (4.1 .46)  as: 


"  (  ^Mt BT)2  -  2RM I  ^e  I  ^Mt BT  + 


The  probabilistic  model  of  Equation  (4.1.44)  can  now  be  extended.  The 
probability  that  a  missile  destroys  an  aircraft  from  a  line  of  sight  distance  r,  with 
aircraft  aspect  angle  AA0p,  and  off  boresight  angle  Aqb  when  a  launch  is 
undetected  is  given  by: 


Pexp(-ad'),  if  d*  >  0 

P(K|r,AASP,A0B,U)  = 

p,  otherwise 
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where 

°e  =  AASP  “  aOB  -  arctan(rsin(AASp).rcos(AASP)  “  tBTva) 
and 

d’=  r  -  ^aTbtCOsCAasp) 

"  (' ^MtBT)2  *  ^ATBTsin(  I  aASP  I ) '  2RM I  °e  1  VMJBT 


+  2Rm2 


1  -  COS 


yMTBT 

rM 


^MtBT 

rM 


^mtbt 

rm 


2|®el 


1  n 


2 


+  2rm2 


-  VMTBT 
'  rM 


r  ^MTBT 

L  RM 


2|Oe 


1 

2 


* 


sin 


^mtbt 

r  ^MtBT  r 

rm 

^  rm 

rM 


-  2\<&c 


1 

2 


1 

2 


The  case  when  a  missile  launch  is  detected  by  the  target  aircraft  will  now 
be  considered.  The  probability  that  a  launch  is  not  detected  is  given  by  Pp  =  1  - 
Py.  The  assumption  for  a  detected  missile  launch  is  that  the  target  aircraft  tries 
to  outrun  the  missile.  The  deterministic  version  for  an  infinitely  maneuverable 
aircraft  was  given  by  Equation  (4.1.42),  which  (rewritten  in  line  of  sight 
coordinates )  is  given  by: 

(4.1.54)  r  =  RMsin(AoB)  +  ( VM  -  ^a)tBT  “  RMAOB- 
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Equation  (4.1.54)  assumed  that  the  target  aircraft  could  instantaneously 
align  its  velocity  vector  outward  along  the  line  of  sight.  Now  an  approximation  will 
modify  this  aquation  to  account  for  the  energy  and  time  needed  for  this  alignment 
by  the  target  aircraft.  The  equations  of  motion  in  line  of  sight  coordinates  are: 

r*  =  -  [  VaCOS(Aasp)  +  \/Mcos(AOB)  j 

1  r  I  VA 

(4.1.55)  Aqb*  =  ~  ^Asin(AASP) +  ^Msin(AOB) - 

r  L  J  RA 

i  r  i 

aASP*  =  -  ^Asin(AASp)  +  VmsNaOB)  “  —  e- 
r  L  J  rM 

where  we  have  introduced  RA  for  the  turn  radius  of  the  aircraft,  with  4>  e  [-1,1] 
being  the  missile  control,  while  the  aircraft  control  9  e  [-1,1].  An  approximation  to 
the  loss  in  distance  along  the  line  of  sight  of  the  aircraft  due  to  its  turning  outward 
(aAsp  =  it)  is  given  by  RA(sin(  |  AASP  | )  -  w  +  |  AASP  |  ) 32 

Similarly  to  the  undetected  missile  case  we  combine  the  results  into  a 
probabilistic  model  as33: 


(4.1.56) 


P(K|r,AASP,AoB.D)  = 


Pexp(-adM),  if  d" 
p,  otherwise 


>0 


where 

d"  =  r -  RM(sin( |  Aqb I )  “  lAOBl)_(v'M“  WbT- 

RA(^  -  !  Aasp  |  -  sin(  |  AASp  |  ))• 


32Note  we  have  assumed  the  missile  is  launched  from  a  distance  such  that  f 1 
«  vA/rA- 

33Note  a  and  p  have  the  same  meaning  as  in  the  undetected  case. 
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Equations  (4.1.53)  and  (4.1.55)  give  a  probabilistic  missile  effectiveness 
model  in  terms  of  nominal  parameters.  This  model  does  not  directly  take  altitude 
into  account  as  in  general  this  is  a  fast  state  variable  that  has  little  effect  as  far  as 
initial  altitude  separation  at  launch.  The  maximum  turn  rates  and  velocities  are 
effected  by  altitude  and  airframe  load  factors.  This  remain  near  a  nominal  level 
for  a  launch  at  given  altitude.  Missiles  such  as  AMRAAM  normally  seek  a 
nominal  mid-flight  altitude  to  optimally  set  these  parameters.  This  model  display 
a  reasonable  approximation  to  actual  missile  firings  sufficient  for  our  purposes. 
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4.1.4.  AIRCRAFT  FRAME  DESCRIPTION 


The  aircraft  frame  and  relational  subframes  can  now  be  described.  The 


details  of  the  attached  procedures  are  found  in  other  sections  such  as  4.1.3, 

4.1.2,  and  6. 

Aircraft  Frame 

Mesarovic  Slot 

Parameters: 


State: 


Max  Clift: 

a  real 

Max  Thrust: 

a  real 

Max  G's: 

a  real 

Weight: 

a  real 

Wing  Surface  Area: 

a  real 

Oswald  Efficiency: 

a  real 

Wing  Aspect  Ratio: 

a  real 

X: 

a  real 

Y: 

a  real 

Z: 

a  real 

VLG: 

a  real 

aLG; 

a  real 

Heading  Rate: 

a  real 

Pitch  Rate: 

a  real 

Heading: 

[-Pi, Pi] 

Pitch: 

(-Pi/2, P 

Fuel: 

a  real 

Number_Air_Air_Missiles: 

[0,1, 2.3 

Number_Air_Ground_Missi: 

10,1,2.3 

Controls: 

Bank: 

Thrust: 

Clift: 

if-needed  (procedure  to  update  state) 
Relational  State  Slot 
Tactic: 


(-Pi/2,  Pi/2) 

(0,  Max_Thrust) 
(0,  Max_Clift) 


Causal  Attrition  Slot 

Air  To  Air  Missile  Type  Attrition: 
Missile  Description: 

Nominal  Burn  TIME: 
Nominal  Velocity: 

“Detected; 

"Detected1 

“Undetected1 


(LOS_Pursuit, 
LOS_Evade, 
LOS_Ground_Pursuit, 
LOS_Ground_Evade 
Harrier_Attack_F  1 6 
Harrier_Evade_f1 6 
FI  6_Evade_Harrier 
F 1 6_Attack_Harrier) 


a  real 
a  real 

a  positive  real 

[0,1] 

a  positive  real 


Det: 


if-needed  (procedure  to  determine  if 
target  destroyed) 

Air  To  Ground  Missile  Type  Attrition: 
Missile  Description: 

Nominal  Burn  TIME: 

Nominal  Velocity: 
a: 

P: 

if-needed  (procedure  to  determine  if 
target  destroyed) 

Self  Attrition  Slot 

if-needed  (procedure  to  test  for  crash) 


[0.1] 


a  real 
a  real 

a  positive  real 
[0,1] 


Harrier  Attack  of  FI  6  Frame 
Harrier  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

F 16  Type  Target: 

FI 6  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

Max  Turn  Radius: 

if-needed  (procedure  to  determine  Harrier  control) 


a  real 
a  real 
a  real 

an  aircraft  Frame 

a  real 
a  real 
a  real 
a  real 


Harrier  Evade  FI  6  Frame 
Harrier  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

FI 6  Type  Target: 

FI 6  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

Max  Turn  Radius: 

if-needed  (procedure  to  determine  Harrier  control) 


a  real 
a  real 
a  real 

an  aircraft  Frame 

a  real 
a  real 
a  real 
a  real 


FI  6  Evade  Harrier  Frame 

Harrier  Type  Target: 
Harrier  Description: 
Speed: 

Kill  Radii: 

Kill  Eccentricity: 
FI 6  Description: 
Speed: 


an  aircraft  Frame 

a  real 
a  real 
a  real 


a  real 


95 


Kill  Radii:  a  real 

Kill  Eccentricity:  a  real 

Max  Turn  Radius:  a  real 

if-needed  (procedure  to  determine  FI  6  control) 


F6  Attack  Harrier  Frame 
Harrier  Type  Target: 

Harrier  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

FI  6  Description: 

Speed: 

Kill  Radii: 

Kill  Eccentricity: 

Max  Turn  Radius: 

Advasary: 

if-needed  (procedure  to  determine  FI  6  control) 


an  aircraft  Frame 

a  real 
a  real 
a  real 

a  real 
a  real 
a  real 
a  real 

Aircraft  Frame 


LOS  Pursuit  Frame 

Advasary:  Aircraft  Frame 

if-needed  (procedure  to  determine  LOS  heading  control) 


LOS  Evade  Frame 

Advasary:  Aircraft  Frame 

if-needed  (procedure  to  determine  LOS  heading  control) 


LOS  Ground  Pursuit  Frame 

Fixed  Altitude:  a  positive  real 

Advasary:  SAM  Frame, 

Ground  Target 

Frame 

if-needed  (procedure  to  determine  LOS  heading  control) 


LOS  Ground  Evade  Frame 

Fixed  Altitude:  a  positive  real 

Advasary:  SAM  Frame, 

Ground  Target  Frame 

if-needed  (procedure  to  determine  LOS  heading) 
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4.2.  SAMS  (Surface  To  Air  Missile  Sites) 

The  next  component  type  to  be  discussed  is  SAM  sites.  The  probabilistic 
model  for  the  missile  capabilities  for  simplicity  is  assumed  to  have  the  same  form 
as  in  section  4.1.3.  The  difference  between  air  to  air,  and  air  to  ground  missiles 
will  be  imbedded  in  nominal  parameters  such  as  Vj^,  Tgj,  Pp,  etc.  The  location 
of  the  SAM  sites  is  fixed  but  may  be  unknown  to  other  components. 
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4.3.  HOMES  (Home  Aircraft  Bases) 


The  Home  component  type  describes  airfields.  Home  components  have 
fixed  location,  and  their  purpose  is  to  reproduce  (takeoff),  or  cause  attrition 
(landing)  of  aircraft  components.  The  frame  is  shown  below. 


HOME  Frame 

Mesarovic  Slot 

Parameters: 

X:  a  real 

Y:  a  real 

State: 

Number_Aircraft_Type_A:  a  pos  integer 

Number_Aircraft_Type_B:  a  pos  integer 

Causal  Attrition  Slot 

if-needed  (procedure  to  land  aircraft) 
Reproductive  Slot 

if-needed  (procedure  to  takeoff 
aircraft) 
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4.4.  GROUND  TARGETS 


The  last  component  type  to  be  considered  are  Ground  Targets. 
Components  of  this  type  are  targets  of  tactical  interest.  Ground  target  real  world 
counterparts  could  be  factories,  or  other  facilities  of  importance.  This  very  simple 
frame  is  shown  below. 


Ground  Target  Frame 
Mesarovic  Slot 

Parameters: 

X:  a  real 

Y:  a  real 
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5.  SYSTEM  INTEGRATION 

This  .  section  outlines  how  to  integrate  all  the  concepts  discussed 
throughout  this  dissertation  together  via  the  semantic  control  paradigm.  The  use 
of  the  probabilistic  missile  models  will  be  discussed,  and  how  to  integrate  the 
results  into  determining  and  using  the  appropriate  differential  game  model. 
Methodologies  for  implementing  the  system  goal  selector  will  also  be  explored 
and  outlined.  The  goal  selector  determines  the  differential  game  models,  and 
from  the  knowledge  base.  Three  possible  methodologies  for  the  goal  selector 
will  be  discussed.  Lastly  the  control  adapter  will  be  discussed.  The  identifier  is 
not  discussed  as  perfect  information  is  assumed. 
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5.1.  FIRING  AND  AVOIDANCE  SURFACES  AS  GOALS 


Two  surfaces  must  be  determined,  a  firing  surface,  and  an  avoidance 
surface.  The  firing  surface  is  a  function  of  resource  allocation  and  a  desired 
probability  of  being  able  to  destroy  all  assigned  targets  for  a  mission.  A  simplified 
equation  for  the  probability  of  being  able  to  destroy  all  assigned  aircraft  targets 
(Pm),  based  on  the  negative  binomial  distribution  is  given  by: 

(x— 1 ) ! 


(5.1) 


N 

Pm  =  I 


x=K  (x-K)l(K-l)! 


pK(l-p)*-^ 


where 

N  =  Number  Of  Missiles  Remaining  On  Aircraft 
p  s  Probability  That  A  Missile  Will  Destroy  Its  Target 
K  s  Number  Of  Targets 
x  s  Number  Of  Missiles  Fired. 


Equation  (5.1)  is  solved  for  p  given  P^,  N  and  K.  A  more  complex 
equation  might  include  a  tradeoff  on  the  cost  of  a  missile,  or  scarcity  of  the 
missiles  as  a  resource.  Suppose  that  an  aircraft  has  6  air  to  air  missiles,  and  the 
expected  number  of  encounters  is  2  aircraft  targets.  To  assure  a  90%  probability 
of  being  able  to  destroy  the  expected  targets,  each  missile  must  be  fired  such 
that  the  missile  has  a  58%  probability  of  destroying  its  target. 

The  next  step  is  to  use  equations  (4.1.53)  and  (4.1.56)  to  determine  the 
firing  surface.  Figures  5.1  and  5.2  show  the  firing  surface  for  a  missile  to  have  a 
58%  probability  of  destroying  its  target  when  Vm  =  1,  Rm  =  1,  Va  =  .25,  Ra  =  4, 
Tbt  =  9,  a  =  0.25,  p  =  0.9,  and  PD  =  0.2. 
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The  avoidance  surface  is  determined  by  the  amount  of  risk  that  a  pilot  or 
the  pilot's  superiors  are  willing  to  accept  We  will  assume  lor  simplicity  that  it  is 
desirable  to  keep  the  probability  of  being  destroyed  by  an  adversarial  missile 
launch  below  a  predetermined  threshold.  Equations  (4.1.53)  and  (4.1.56)  are 
used  again,  this  time  to  determine  the  avoidance  surface.  The  role  of  aircraft  and 
missile  parameters  are  now  interchanged  to  indicate  ownship  aircraft  capabilities 
versus  the  adversaries  missile  capabilities.  Figures  5.3  and  5.4  show  an 
avoidance  surface  for  an  adversary  missile  kill  probability  of  10%  percent  when 
Vm  =  1,  Rm  =  1,  Va  =  .33,  Ra  =  2,  Tbt  =  7,  a  =  0.75,  fJ  =  0.7,  and  Pp  =  0.4. 


FIGURE  5.3 

Missile  Range  Avoidance  Surface  For  Prob  Kill  Of  0.10 
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5.2  GOAL  SELECTOR 

Once  the  firing  and  avoidance  surfaces  are  determined,  the  problem 
becomes  one  of  determining  trajectories  and  controls  that  allow  the  ownship 
aircraft  to  reach  the  firing  surface  without  penetrating  the  avoidance  surface.  The 
goal  selector  of  the  semantic  control  paradigm  must  identify  the  differential  game, 
parameters  and  role  that  the  ownship  aircraft  should  assume  from  the  knowledge 
base  of  differential  games.  Three  alternatives  for  the  system  goal  selector  expert 
system  will  be  explored,  rule  based,  neural  nets,  and  The  Analytical  Hierarchy 
Process. 

One  key  requirement  for  the  "expert  system"  implementation  of  the  goal 
selector  is  that  the  knowledge  (i.e.  the  differential  game  library)  must  be 
separated  from  how  the  knowledge  is  manipulated.  It  is  imperative  that  the 
capability  exist  to  add  new  differential  games  to  the  knowledge  base  for  the 
possibility  of  improved  performance,  without  modifying  the  system  as  a  whole. 
This  feature  is  known  as  "incremental  development"34.  "Incremental 
development"  allows  the  maintenance  and  improvement  of  performance  by 
incrementing  the  knowledge  base  as  more  is  learned  about  ownship  and 
adversarial  performance  via  the  models  incorporated  into  the  knowledge  base. 
Without  "incremental  development"  a  new  model,  or  improved  adversarial 
performance  could  mean  the  obsolescence  or  complete  redesign  of  the  semantic 
control  system. 

A  "shell"  is  a  tool  for  building  a  limited  set  of  knowledge  base  applications 
based  on  generalizing  the  requirements  common  among  these  applications.  The 


^Lazarev,  G.  L.  pg  1 34 
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"shell"  serves  to  separate  the  knowledge  represented  and  the  "inference  engine". 
The  "inference  engine"  is  primarily  responsible  for  control.  The  "inference 
engine"  must  also  handle  features  such  as  user  interaction,  explanation 
capabilities,  and  uncertainty.  Our  goal  is  to  outline  how  to  build  the  expert 
system  "shell"  for  the  goal  selector  such  that  "incremental  development"  is 
supported,  and  performance  will  approach  optimality  for  the  current  differential 
games  in  the  knowledge  base. 

5.2.1  RULE  BASED  APPROACH 

The  first  methodology  to  be  considered  for  building  the  differential  game 
expert  "shell"  will  be  a  rule  based  system.  It  should  be  noted  that  the  majority  of 
existing  expert  systems  are  rule  based  due  to  their  naturalness,  ease,  and 
uniformity  of  expression.  Rule  base  systems  are  of  the  form 

If  condition(s)  then  action(s) 

These  rules  can  be  used  in  a  forward  or  backward  chaining  fashion. 
Forward  chaining  starts  with  a  rule  base  and  a  set  of  known  facts.  A  rule  that 
matches  the  current  set  of  known  facts  is  triggered.  The  set  of  known  facts  is 
modified  by  the  triggered  rule’s  action.  The  process  is  repeated  for  the  set  of 
updated  facts  until  a  solution,  or  a  failure  is  reported.  Reasoning  is  performed 
from  facts,  using  rules  until  a  solution  is  found. 

Backward  chaining  is  goal  driven  processing.  Reasoning  is  performed 
from  a  goal  using  rules  to  existing  facts.  Rules  are  used  from  actions  to 
conditions.  Each  hypothesized  goal  is  reduced  to  a  set  of  sub-goal.  The  process 
is  then  repeated  until  the  reduced  sub-goal  can  be  identified  with  the  existing 
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facts.  One  of  the  main  uses  of  backward  chaining  is  the  generation  of 
explanations  for  a  decision. 

There  are  two  reasons  we  give  only  brief  consideration  to  the  rule  based 
approach.  Generating  general  rules  splicing  the  differential  games  together 
appears  to  demand  understanding  the  dynamics  of  the  encounters  at  the  same 
level  that  prevents  the  solution  of  higher  order  games.  Rule  based  approaches 
also  tend  to  be  brittle.  An  increment  in  the  knowledge  base  could  change  a  set  of 
complex  rules  drastically.  Generating  a  set  of  rules  for  a  specific  set  of 
differential  games  would  create  a  non-robust  goal  selector.  Rule  based 
approaches  do  not  appear  promising. 

5.2.2  NEURAL  NET  APPROACH 

The  next  expert  system  type  to  be  considered  is  the  Artificial  Neural  Net 
expert  system  as  proposed  by  Gallant.  Interest  in  Artificial  Neural  Nets  has  had 
extremes  in  popularity  since  the  early  1960s.  Artificial  Neural  Nets  are 
computational  mechanisms  based  on  the  structure  of  the  brain.  Currently  there 
is  an  explosion  of  interest  and  research  in  Artificial  Neural  Nets  and  their 
capabilities.  This  explosion  follows  a  long  winter  of  interest  in  Artificial  Neural 
Nets  during  the  1 970s. 

Initially,  during  the  1960s,  there  was  a  great  deal  of  excitement  over  linear 
neuron-like  models  termed  perceptrons33.  Minsky  and  Papert  demonstrated  in 
their  book  Perceptrons  the  key  limitation  that  perceptrons  could  only  solve 
linearly  separable  problems.  This  revelation  lead  to  the  wide  scale  abandonment 
of  Artificial  Neural  Nets  in  the  1970s  and  to  the  flow  of  research  dollars  to 


35Rosenblatt 
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Artificial  Intelligence  (Al).  Recent  advances  with  multilayered  networks  and  new 
network  learning  paradigms  such  as  backpropagation  have  overcome  the 
limitation  of  linear  separability  and  caused  a  major  resurgence  in  artificial  neural 
net  research. 


Hecht-Nielsen36  gives  the  following  general  definition  of  an  artificial  neural 
net  system  [ANS]: 

A  neural  network  is  a  parallel,  distributed  information 
processing  structure  consisting  of  processing  elements  (which 
can  possess  a  local  memory  and  carry  out  localized  information 
processing  operations)  interconnected  together  with 
unidirectional  signal  channels  called  connections.  Each 
processing  element  has  a  single  output  connection  which 
branches  ("fans  out")  into  as  many  collateral  connections  as 
desired  (each  carrying  the  same  signal  -  the  processing  element 
output  signal.)  The  processing  element  output  signal  can  be 
any  mathematical  type  desired.  All  of  the  processing  that  goes 
on  within  each  processing  element  must  be  completely  local; 
i.e.,  it  must  depend  only  upon  the  current  values  of  the  input 
signal  arriving  at  the  processing  element  via  impinging 
connections  and  upon  values  stored  in  the  processing  element's 
local  memory. 

Simpson37  gives  a  simpler,  less  rigorous  definition,  "...  an  ANS  is  a  nonlinear 
directed  graph  with  edges  that  is  able  to  store  patterns  by  changing  the  edge 
weights  and  is  able  to  recall  patterns  from  incomplete  and  unknown  inputs". 


A  biological  neuron  is  the  basic  building  block  of  the  nervous  system. 
Figure  5.5  shows  a  simplified  view  of  a  biological  neuron.  The  body  cell  of  the 
neuron  is  termed  the  "soma".  Connected  to  the  "soma"  are  multiple  "dendrites" 
and  an  "axon",  these  serve  as  the  mechanism  of  communication  to  other 
neurons.  The  "dendrites"  are  spine  like  connections  which  receive  stimulus  from 
other  neurons.  Each  neuron  has  a  single  "axon"  that  serves  to  transmit  the  same 


36Simpson  pg  3 

^Simpson  pg  4 
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stimulus  to  all  the  other  connected  neurons.  The  connection  between  a  neuron's 
"axon"  and  another  neuron’s  "dendrite"  is  termed  a  "synapse".  Each  "synapse" 
has  a  level  of  transmission  of  the  stimulus  on  the  "axon"  to  the  connected 
"dendrite".  Only  when  a  neuron  receives  sufficient  stimulus  from  other  neurons 
connected  at  the  "dendrites"  does  the  neuron  become  active  and  send  a  stimulus 
out  on  its  own  "axon".  Since  each  neuron  accepts  a  different  level  of  stimulus 
from  a  connected  neuron  based  on  the  transmission  level  at  the  "synapse",  the 
network  knowledge  is  equivalent  to  these  transmission  levels.  Biological  neural 
networks  work  by  a  biochemical  process  that  is  beyond  our  scope  of  interest 
here. 


_ FIGURE  5.5 

Biological  Neuron 


Soma 


The  basic  model  for  an  artificial  neuron  is  shown  in  Figure  5.6.  Lines  (or 
wires)  replace  the  functions  of  biological  "axon"  and  "dendrites".  "Synapses"  are 
replaced  by  weights  (or  resistors).  The  body  of  the  neuron  or  "soma"  is  split  into 
two  components.  The  first  component  is  an  adder  that  sums  up  the  weighted 
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outputs  of  other  neurons.  The  second  component  is  the  activation  function.  This 
function  determines  the  artificial  neuron’s  activation  level  based  on  weighted  net 
input.  Typically  this  function  is  an  s-shaped  function  known  as  a  squashing 
function. 


_ FIGURE  5.6 

Artificial  Neuron  Model 


The  method  of  connecting  the  network  neurons  defines  the  network 
topology.  Two  general  categories  for  networks  exist,  recurrent  and  non¬ 
recurrent.  Recurrent  networks  have  cycles  in  the  network  connections.  These 
cycles  make  recurrent  network  dynamical  systems.  Our  interest  lies  in  non¬ 
recurrent  networks.  Non-recurrent  networks  have  no  cycles.  A  non-recurrent 
network  can  be  viewed  as  a  transformation  from  n-dimensional  euclidean  space 
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to  m-dimensional  euclidean  space.  The  special  instance  of  a  non-recurrent  net 
with  which  we  concern  ourselves  is  the  feedforward  net  shown  in  Figure  5.7. 

_ FIGURE  5.7 _ 

Feedforward  Neural  Net 

Output  Layer 

Hidden  Layers 

Input  Layer 


A  feedforward  net  consists  of  a  number  of  layers  of  neurons.  Neurons  in 
one  layer  are  restricted  to  outputting  only  to  neurons  in  the  next  layer,  and 
inputting  only  from  neurons  in  the  previous  layer.  There  is  always  an  input  layer 
which  simply  serves  to  buffer  the  inputs.  An  output  layer  also  exists  and  its 
activation  activation  function  f()  can  be  linear  or  nonlinear.  There  can  also  be  a 
number  of  hidden  layers.  These  hidden  layers  lie  between  the  input  and  output 
layers  and  have  a  non-linear  activation  function  f().  The  power  of  the  feedforward 
net  lies  in  the  non-linear  nature  of  the  hidden  layers.  Recent  research  has  shown 
that  any  measurable  function  can  be  approximated  almost  everywhere  by  a 
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feedforward  net  with  one  hidden  layer  38  The  network  connection  weights  can 
be  programmed  a  priori.  The  weights  are  more  commonly  learned  recursively. 
Different  learning  paradigms  can  be  found  in  Simpson  (1 989). 

Gallant  has  demonstrated  a  methodology  to  use  artificial  neural  networks 
to  build  classification  expert  systems  where  the  outputs  from  the  system  can  be 
represented  by  variables  that  take  on  values  from  a  finite  set.  Each  variable  has 
its  own  finite  set  of  possible  values.  The  particular  artificial  neural  network 
implementation  for  our  interests  will  have  a  Boolean  (two  value)  classification 
system.  Classification  expert  systems  can  handle  a  wide  variety  of  applications. 
Suppose  a  particular  variable  X  can  take  on  continuous  values  in  the  range  [0,1], 
X  could  be  approximated  by  several  Boolean  choice  variables  X-|  ,X2,X3,  where 
each  Xj  corresponds  to  a  range  of  values?39 

if  X  >  1/4,  then  .  1 ;  else  Xt  =  -1  ; 

if  X  >  1/2,  then  X2  =  1 ;  else  X2  = -1 ; 

if  X  >  3/4,  then  X3  =  1 ;  else  X3  =  -1 ; 

so  that  X  =  .6  would  correspond  to  X-|  =  X2  =  1 ,  X3  =  -1 . 

The  artificial  neural  net  for  the  classification  expert  system  shall  have  the 
architecture  described  below.  The  net  architecture  shall  be  feedforward  with  one 
hidden  layer.  The  neurons  are  perceptrons  having  discrete  activation  levels,  -1, 
0  or  1,  corresponding  to  the  Boolean  values  False,  Unknown  or  True 


^Hornik 
"Gallant  pg  1 54 
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respectively.  The  neuron  activation  function  relating  the  activation  of  neurons 
(uk_1  j)  on  layer  k-1  to  neurons  (uk-,)  on  layer  k  is  given  by: 


netkj  =  Z  vv^jju^1  j 


if  netkj  >  0 
if  netkj  =  0 
if  netkj  <  0, 


where  w^j  is  the  weight  connecting  neuron  i  on  layer  k  to  neuron  j  on  layer  k-1 . 


The  name  of  each  neuron  corresponding  to  each  variable  of  interest  must 
also  be  specified.  The  variables  of  interest  must  also  be  partitioned  into 
dependent  and  independent  classes.  The  independent  variables  of  interest  are 
assigned  to  the  input  layer,  while  the  dependent  variables  are  assigned  to  the 
output  layer.40  Partial  knowledge  about  the  relation  between  dependent  and 
independent  variables  can  be  directly  incorporated  in  the  net  by  allowing  only 
partial  connectivity  through  the  hidden  layer  between  sets  of  input  and  output 
neurons.  The  values  of  the  weights  can  be  directly  programmed  if  the  relations 
are  actually  known,  or  more  realistically  they  can  be  learned  from  a  training  set  of 
input  and  output  pairs. 


40This  partitioning  differs  slightly  from  that  proposed  by  Gallant.  Gallant’s 
method  was  to  list  all  the  dependencies  (adjusting  to  avoid  cycles)  having  no 
particular  input,  output,  and  intermediate  layers.  The  number  of  layers 
corresponded  to  the  minimum  number  of  flow  forward  dependencies.  (Gallant  pg 
156) 
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Our  interest  in  classification  systems  will  be  to  determine  what  models  to 
use,  and  the  particular  parameterization  of  the  models  chosen.  The  problem  is 
divided  up  hierarchically.  One  classification  network  is  designed  to  determine 
what  differential  game  and  models  should  be  used,  along  with  pursuer  and 
evader  role  assignments.  Another  set  of  classification  nets  is  also  specified,  one 
net  for  each  output  combination  from  the  first  net.  A  net  that  corresponds  to  an 
output  combination  determines  the  parameterizations  and  flight  characteristics  for 
the  model,  differential  game,  and  role  chosen. 

Figure  5.8  illustrates  the  classification  net  that  determines  which 
differential  game  from  the  knowledge  base  (currently  in  our  stage  I  simulator)  will 
be  chosen  along  with  some  of  the  criteria  that  might  be  used  to  make  the 
decision.  Figure  5.9  illustrates  the  net  corresponding  to  the  "Harrier  Pursuit" 
output  of  the  differential  game  classification  net  shown  in  Figure  5.8.  The  net 
shown  in  Figure  5.9  determines  the  parametrization  of  the  FI  6  pursuit  of  Harrier 
differential  game.  A  game  parameterization  classification  net  exists 
corresponding  to  each  of  the  output  classifications  of  the  net  shown  in  Figure  5.8. 

The  question  of  determining  the  weights  of  the  classification  nets 
discussed  above  and  shown  in  Figures  5.8  and  5.9  still  remains  open.  Generally 
the  connection  weights  are  learned  recursively  via  training  by  one  of  the  methods 
discussed  in  Simpson.  Training  is  generally  done  from  a  training  set  of  desired 
input  output  matchings  for  asymptotic  cases.  The  goal  is  to  leam  the  unknown 
underlying  mapping  from  input  to  output  space  so  that  correct  outputs  will  be  put 
forth  for  inputs  not  in  the  training  set. 
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FIGURE  5.8 
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The  generation  of  a  representative  training  set  is  typically  non-trivial. 
Training  sets  must  be  generated  for  the  differential  game  classification  neural 
net,  and  for  each  of  the  individual  parametrization  classification  neural  nets.  Two 
methods  can  be  considered  for  the  generation  of  the  training  sets:  analysis  and 
simulation. 

Generating  the  training  set  by  analysis  shares  the  main  difficulty  of  the  rule 
based  methodology:  the  requirement  for  complete  mathematical  understanding 
of  how  all  the  differential  games  perform  relatively  for  each  member  of  the 
training  set.41  It  may  be  possible  to  generate  analytically  members  of  the 
training  set  for  a  few  asymptotic  cases,  but  the  their  generation  for  every  case  by 
this  methodology  is  not  really  practical. 

Simulation  methodologies  appear  to  be  the  most  promising  means  of 
generating  a  training  set.  Each  of  the  differential  game  parametrization 
classification  neural  nets  can  be  trained  independently  by  closing  the  loop 
simulating  forward,  and  adjusting  the  weights  by  propagating  the  errors  backward 
in  time.  Once  each  of  the  individual  differential  game  parametrization 
classification  neural  nets  is  trained,  the  differential  game  choice  classification 
neural  net  (Figure  5.8)  training  set  can  be  generated  via  comparative  simulation. 
This  methodology  involves  the  generation  of  the  appropriate  neural  net  output  for 
a  given  neural  net  input  by  searching  all  possible  output  via  simulation  for  the 
best  performance.  Extensive  search  via  simulation  would  of  course  be  extremely 
computationally  intensive.  Therefore  heuristic  search  methods  might  offer 
improvement  in  search  speed.  The  neural  nets  once  trained  on  a  large  enough 

41  It  is  this  difficulty  the  prevents  the  generation  of  solutions  to  higher  order 
differential  games. 
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set,  would  be  expected  to  generalize  well  to  cases  not  directly  included  in  the 
training  set. 


One  important  requirement  is  that  the  expert  system  be  able  to  justify  its 
decisions  to  the  user.  A  system  to  aid  pilots  must  be  able  to  explain  decisions  to 
the  pilot  to  find  acceptance.  Gallant  gives  a  methodology  to  generate 
explanations  of  decisions  via  the  generation  of  if-then  rules  from  classification 
neural  nets.42  His  methodology  is  as  follows: 


i)  List  all  inputs 
discriminant  of  Z;k 


_k-1 


“J 


that  have  contributed  positively  to 
Figure  5.10  generates  Zik_1,  Z2k'1 , 


the 

and 


ii)  Arrange  the  list  bv  decreasing  absolute  value  of  the  weights. 
Figure  5.10  gives  Z^~^ ,  Z5k_1 ,  and  Z-|k_1 . 


iii)  Generate  clauses  for  an  if-then  rule  from  the  ordered  list  from 
ii)  until 


S 

used  for  clause 


> 


?.  7k  i^ji 

remaining  Z*j 


is  satisfied.  The  generated  if-then  rule  for  Figure  5.1 0  given  by 
this  methodology  is: 

if  Z2k_1  is  False  and  Zs^1  is  True 
then  Conclude  That  Zjk  is  True 


42Gallant  pg  1 63 
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Gallant’s  methodology  generates  as  set  of  if-then  rules  for  layer  to  layer 
connections  that  can  be  used  in  a  backward-chaining  fashion  to  explain 
decisions. 

In  conclusion  we  has  shown  in  this  section  that  artificial  neural  nets  are 
candidates  for  implementing  the  semantic  controller  identifier  for  air-combat. 
Such  a  methodology  would  provide  a  real-time  mechanism  for  determining  what 
differential  games  and  parameters  should  be  chosen  from  the  knowledge  base. 
The  main  drawback  of  this  approach  is  the  difficulty  in  obtaining  training  sets. 


_ FIGURE  5.10 _ 

Explanations  by  if-then  Rules 


True 


Rule: 

If  Z*"  False  and  Z*  ■  True, 
Then  Conclude  Z*  is  False 
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5.2.3  ANALYTICAL  HIERARCHY  APPROACH 

The  final  approach  to  be  discussed  for  implementing  the  identifier  is  the 
Analytical  Hierarchy  Process  originated  by  T.L.  Saaty  of  the  Wharton  School. 
The  Analytic  Hierarchy  Process  (AHP)  is  a  general  theory  of  measurement  that 
can  be  used  in  multicriteria  decision  making.  Ratio  scales  are  derived  from 
paired  comparisons.  To  use  AHP  in  a  decision  problem,  a  hierarchic  structure 
that  represents  the  problem  is  needed,  along  with  pairwise  comparisons  to 
establish  relations  within  the  structure. 

Figure  5.11  shows  an  example  of  a  hierarchic  structure  for  a  simple 
example  of  choosing  the  best  high  school  for  a  student  to  attend.  This  simple 
hierarchy  has  the  overall  goal  as  the  apex  (choose  the  school  that  gives  the  most 
overall  satisfaction).  The  criteria  for  selecting  falls  on  the  second  layer.  The 
lowest  layer  contains  the  alternative  (the  school)  choices.  More  complex 
decision  problems  that  have  criteria  and  sub-criteria  would  have  more  layers  in 
the  hierarchy.  Figure  5.12  shows  the  pairwise  comparison  of  the  importance  of 
the  criteria  with  respect  to  satisfaction  with  school.  The  relative  importance  of 
each  of  the  criteria  can  be  shown  to  be  the  principal  eigenvector  of  the  matrix  of 
pairwise  comparisons.  Pairwise  comparison  would  be  repeated  for  each  school 
choice  with  respect  to  each  of  the  criteria.  The  set  of  principal  eigenvectors  for 
each  of  the  comparison  matrices  would  finally  be  composed  via  matrix 
multiplication  to  give  an  overall  ranking  of  the  merits  of  each  alternative  (school 
choice). 
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The  AHP  has  a  formal  mathematical  basis.  We  first  present  some  partially 
ordered  set  notation. 

Definition  5.1 :  Let  S  be  a  partially  ordered  set  (P.O.S).  Then  for  x,y  e  S: 

i)  x  <  y  implies  x  <  y  and  x  *  y 

ii)  y  covers  x,  if  x  <  y  and  there  does  not  exist 
t  e  S  such  that  x  <  t  <  y 

iii)  x'  =  {  y  |  x  covers  y} 

iv)  x+  =  {  y  |  y  covers  x} 


_ FIGURE  5.11 _ 

HIERARCHY  FOR  HIGH  SCHOOL  CHOICE 


Satisfaction  With  School 


Developed  For  Saaty’s  Son 
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FIGURE  5.12 

Comparison  of  characteristics  with  respect  to  overall  satisfaction  with  school 
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Given  our  P.O.S  notation  a  Hierarchy  can  formally  be  defined.  The 
following  definition  introduces  the  concept  of  layers  for  a  P.O.S. 


Definition  5.2:  Let  H  be  a  finite  P.O.S.,  H  is  a  hierarchy  if  the  following  conditions 
are  satisfied; 


i)  There  exist  a  partition  of  H  into  set  Lu, 

k  =  1 . h  where  L-j  =  {b},  b  is  a  single  element 

(apex). 

ii)  x  e  Lj<  implies  x'  is  a  subset  of  L^+-j , 
k  =  1,...,  h-1 


iii) 


x  e  Lj<  implies  x+  is  a  subset  of 
k  =  2,...,  h 


Lk-1. 


We  now  lay  out  the  ground  work  for  determining  rankings  of  the  elements 
of  a  layer  with  respect  to  each  member  of  the  next  higher  layer.  Let  U  be  a  finite 
set  of  "alternatives".  Let  C  be  a  set  of  criteria  with  respect  to  which  the  elements 
in  U  are  compared.  For  c  e  C,  let  Pq:  UxU  R+  such  that  Pc(Aj,Aj)  =  ay  e  R* 
for  every  Aj,Aj  e  l/.4^  The  function  Pq  is  of  course  the  pairwise  comparison. 
Definition  5.3:  For  every  c  e  C,  (UxU,R*,Pq)  is  called  a  "primitive  scale". 


is  the  set  of  positive  reals 
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Definition  5.4:  For  Aj.Aj  e  U  andce  C: 

i)  Aj  >q  Aj  iff  P0(Aj,Aj)  >  1  ”Aj  dominates  Aj  wrt  c" 

ii)  Aj  »q  Aj  iff  PQ(Aj.Aj)  =  1  "indifference" 

Reciprocal  consistency  is  imposed  on  the  pairwise  comparisons.  It  is  imposed  so 
that  if  A  is  twice  as  important  as  B,  then  B  is  half  as  important  as  A.  Saaty’s  first 
axiom  states  this  condition. 

Axiom  5.5:  For  every  Aj.Aj  e  Uand  c  e  C,  Pq(Aj.Aj)  =  1/PQ(Aj,Aj). 

Our  main  sub-objective  at  this  point  is  this:  given  a  matrix  A  =  (ajj)  = 
(Pc(Aj.Aj))  of  pairwise  comparisons  obtain  a  scale  of  rank  order  of  the 
alternatives.  Let  RM(n)  be  the  set  (nxn)  positive  reciprocal  matrices,  and 
define  a  function  W:R^(nj->[0,1]n  which  maps  (nxn)  reciprocal  matrices  to  n- 
dimensional  vectors  with  positive  components  with  magnitude  less  than  or  equal 
to  one.  The  function  W  determines  from  a  matrix  of  pairwise  comparisons  the 
relative  rankings  of  the  alternatives  based  on  the  relative  magnitude  of  the 
corresponding  components  of  the  vector. 

Definition  5.6:  (R^(n),[0,1]n,W)  is  the  "derived"  scale. 

There  are  of  course  many  different  "derived"  scales.  Saaty  built  his 
"derived"  scale  around  the  notion  of  consistency.  Consistency  is  the  property  of 
transitivity  in  the  pairwise  comparisons. 

Definition  5.7:  The  mapping  Pq  is  said  to  be  consistent  iff, 

Pc(Ai’Aj)pC<Aj-Ak)  =  pdAi-Ak) for  everV  '*i*k 
or 

ajjajk  =  ajk  for  every  i,j,k 
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While  the  pairwise  comparisons  are  not  required  to  be  totally  consistent 
(they  seldom  are),  the  "derived"  scale  is  based  on  consistency  for  the  idealized 
case.  Small  perturbations  in  consistency  can  be  likened  to  noise  in  a 
measurement.  Letting  Wj  be  the  ith  component  of  the  derived  scale  vector  W, 
then  if  A  were  totally  consistent  then  ajj  =  Wj/wj  for  every  i,j  =  1,2,...,n.  This 
implies  the  following: 


n 

I 

H 


ayWj/Wj  =  n 


n 

£  Vi  -  nwi 


AW  =  nW 


Thus  W  is  the  eigenvector  corresponding  to  the  maximum  eigenvalue  n.  Small 
perturbations  of  consistency  implies  by  the  continuity  of  the  eigenvalues  and 
eigenvectors  that: 

AW  =  rmaxw- 

where  rmax  is  the  maximum  eigenvalue  of  A,  and  W  is  the  corresponding 
eigenvector. 

At  this  point  we  have  shown  how  Saaty's  methodology  relates  alternatives 
on  one  layer  of  the  hierarchy  to  each  of  the  criteria  on  the  preceding  layer.  It  still 
is  to  be  shown  how  to  relate  alternatives  to  criteria  on  layers  not  directly  adjacent, 
in  particular  to  relate  alternatives  on  the  bottom  layer  to  the  goal  on  the  apex  of 
the  hierarchy. 

Recall  the  notation  of  definitions  5.1  and  5.2.  Assume  that  the  "derived" 
scales  (normalized  principal  eigenvectors  corresponding  to  the  pairwise 
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comparison  matrixes)  have  been  defined  for  every  member  of  the  hierarchy  (x  e 
H)  ranking  all  member  of  the  next  level  of  the  hierarchy  (X")  with  respect  to  x,  i.e., 
for  every  x  e  H  a  function, 

WX:X"  ->[0,1]  such  that  I  Wx(y)  =  l, 

yeX- 

is  defined.  Given  any  element  x  e  of  a  particular  level  a  of  the  hierarchy,  and 
subset  S  of  a  lower  level  Lp  (a<  (3),  Saaty  defined  a  function  Wx  s:S  ->  [0,1] 
which  reflects  the  properties  of  the  "derived"  scales  of  the  intermediate  levels  L^, 

k  =  a, ...,(3-1 .  Let  X  =  {x1  ,...,xmk+1}  =  LK+1 ;  Y  =  {y-, . ymk}  =  LK,  and  assume 

there  exist  z  e  L^_i  such  that  Y  =  z*,  then  the  function  YJz  X:X  ->  [0,1]  relating 
hierarchy  layer  LK_-|  to  layer  Lk+i  can  be  constructed  as  follows: 

wz,X<Xi)  *  |K  Wyj(Xi)Wz(yj);  i  =  1 . mK+1. 

The  matrix  B  is  formed  by  letting  by  =  Wyj(x,),  defining  W-,  =  Wz  X(Xj)  and  Wj  = 
Wz(yp  then: 

Wj  =  .IK  bjjW'j ;  i-1 . mK+1 

W  =  BW 


This  derivation  relates  layer  k-1  to  layer  k+1  by  a  simple  matrix  multiplication,  and 
is  generalized  in  the  following  theorem. 

Theorem  5.8:  Let  H  be  a  hierarchy  with  largest  element  b  and  h  levels.  Let  B^  be 
the  priority  matrix  of  the  k**1  level,  k  =  1,...,h,  if  W  is  the  priority  vector  of  the  pth 
level  with  respect  to  some  z  e  Lp_-j ,  then  the  priority  vector  W  of  the  q*h  level  (p 
<  q)  with  respect  to  z  is: 
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W  =  BqBq_i  ...Bp+i  w’, 
in  particular  with  respect  to  b 

W  =  BhBh_1...B2b1 

The  power  of  the  AHP  has  been  shown  to  be  the  structure  it  brings  to 
multi-criteria  decision  problems,  along  with  the  necessity  to  make  only  pairwise 
comparisons  among  alternatives.  It  is  much  easier  to  compare  two  objects  at  a 
time,  then  to  rate  a  whole  group.  The  AHP  mechanism  has  the  power  to 
compose  these  local  rankings  into  the  hierarchic  structure  and  produce  an  overall 
relative  ranking  of  the  merits  of  the  bottom  level  alternatives  with  respect  the 
overall  goal. 

Our  interest  is  to  incorporate  the  AHP  as  a  mechanism  to  implement  the 
system  identifier.  It  is  our  goal  here  to  point  out  only  how  the  identifier  might  be 
formulated  to  use  the  AHP,  along  with  projected  strengths  and  weakness. 

A  hierarchy  is  created  indicating  criteria  and  sub-criteria  for  choosing  the 
most  appropriate  differential  game  and  parameterization.  Figure  5.13  shows  a 
sample  three  layer  hierarchy  for  choosing  the  differential  game.  The  criteria  in 
this  example  include  fuel  usage,  missile  usage,  lethality  of  ownship  against 
adversary,  risk  of  ownship  from  adversary,  duration  of  the  encounter,  and 
dynamic  sensitivity  of  the  differential  game  model.  Dynamic  sensitivity  of  the 
differential  game  model  measures  how  much  extra  headroom  a  model  has  to  the 
dynamics  of  the  adversary  before  the  model  is  invalidated  by  penetration  of  the 
barriers  or  other  critical  assumptions  of  the  underlying  model  are  invalidated. 
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FIGURE  5.13 

EXAMPLE  AHP  HIERARCHY  FOR  CHOOSING  MOST  APPROPRIATE 
DIFFERENTIAL  GAME  AND  PARAMETERIZATION 


BEST  DIFFERENTIAL  GAME  AND  PARAMETERIZATION 


A  pairwise  comparison  matrix  is  continually  updated  for  each  layer  of  the 
hierarchy  with  respect  to  each  of  the  criteria  of  the  previous  layer.  One 
modification  to  the  use  of  the  AHP  is  made  here  by  following  the  method  of 
Forman  (1987).  Instead  of  directly  comparing  the  differential  game  alternatives, 
levels  are  designated  for  each  of  the  bottom  layer  criteria.  An  example  of  these 
levels  is  shown  in  Figure  5.13. 

A  comparison  matrix  for  the  levels  is  created  only  with  respect  to  the 
criteria  for  which  the  levels  were  designated,  enabling  a  priority  vector  to  be 
determined.  A  priority  vector  relating  all  the  levels  to  a  particular  criterion  can  be 
created  by  augmenting  the  original  priority  vector  with  zeroes  for  non-related 
levels.  Figure  5.15  illustrates  the  comparison  matrix  and  priority  vectors  for  the 
fuel  levels  of  Figure  5.13.  Once  the  augmented  comparison  vectors  have  been 
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determined  with  respect  to  each  bottom  level  criterion,  the  overall  priority  vector 
of  the  levels  can  be  determined  by  the  normal  AHP  composition  technique  of 
Theorem  5.8.  Rgure  5.16  shows  an  example  of  the  priority  vectors  for  the  levels 
of  each  criterion,  the  priority  vector  of  the  criteria,  and  the  overall  composed 

priority  vector  of  the  levels  with  respect  to  the  goal  of  choosing  the  best 

differential  game.  The  pairwise  comparison  matrices  were  omitted  for  brevity. 

A  scoring  technique  is  used  to  choose  the  differential  game  and 

parameterization  from  the  knowledge  base.  A  level  under  each  criterion  is 
determined  for  each  differential  game  alternative.  The  score  consists  of  the  sum 
of  the  weights  corresponding  to  the  designated  levels.  Assume  for  our  simple 
example  that  there  are  two  differential  games  and  each  game  has  two 

pcrameterizations  which  have  had  levels  determined  for  the  criteria  as  shown 
below.  The  respective  scores  using  the  values  of  the  overall  priority  vector  of 
Figure  5.16  are  also  determined  and  are  shown  in  Figure  5.14. 

Differential  Game  1  Parametization  2  has  the  highest  score  and  would  be 
the  differential  game  chosen.  It  is  important  to  note  that  the  criteria  priority 
vectors,  pairwise  comparisons,  and  levels  would  be  dynamically  changing  during 
an  air-combat  mission.  Fuel  might  have  a  low  priority  early  in  a  mission,  but  later 
might  have  high  priority,  once  a  shortage  develops. 


127 


FIGURE  5.14 

_ EXAMPLE  SCORING _ 

Diff  Game  1  Param  1  (Score:  0.375) 

High_S  Two  Short_E  Low_F  Low_R  Med_L 
0.035  +  0.020  +  0.043  +  0.045  +  0.196  +  0.036  =  0.375 
Diff  Game  1  Param  2  (Score:  0.434) 

Los_S  One  Med_E  Med_F  Med_R  High_L 
0.138  +  0.080  +  0.023  +  0.023  +  0.098  +  0.072  =  0.434 
Diff  Game  2  Param  1  (Score:  0.335) 

Very_High_S  One  Med_E  Low_F  Med_R  High_L 
0.017  +  0.080  +  0.023  +  0.045  +  0.098  +  0.072  =  0.335 
Diff  Game  2  Param  2  (Score:  0.316) 

Medium_S  >Two  Long_E  Low_F  Low_R  Low_L 
0.035  +  0.010  +  0.012  +  0.045  +  0.196  +  0.018  =  0.316 


FIGURE  5.15 

Level  Comparisons  With  Respect  To  Fuel  Usage 


High_F 

Medium_F 

Low_F 

High_F 

1 

1/2 

1/4 

Medium_F 

2 

1 

1/2 

Low_F 

4 

2 

1 

"Priority  Vector  Of  Fuel  Usage  Levels" 

High_F  0.143 

Medium_F  0.286 
Low  F  0.571 
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"Augmented  Priority  Vector  Of  Levels  WRT  Fuel  Usage" 


Very_High_S 

0.0 

Medium_E 

0.0 

Short_E 

0.0 

Low_F 

0.571 

Medium  F 

0.286 

High  F 

0.143 

High_R 

0.0 

Medium_R 

0.0 

Low_R 

0.0 

Low  L 

0.0 

FIGURE  5.16 

Priority  Vectors  For  Example  Hierarchy 


"Priority  Vector  Criteria  With  Respect  To  Goal  Of  Choosing  Most  Appropriate 

Differential  Game" 


Dynamic  Sensitivity  0.259 

Missile  Usage  0.110 

Encounter  Duration  0.084 

Fuel  Usage  0.079 

Risk  0.343 

Lethality  0.125 


"Priority  Vector  Of  Dynamic  Sensitivity  Levels" 

Very_High_S  0.067 

Hign_S  0.133 

Med_S  0.267 

Low  S  0.533 
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"Priority  Vector  Of  Missile  Usage  Levels" 

0.727 
0.182 
0.091 


One 
Two 
>  Two 


"Priority  Vector  Of  Encounter  Duration  Levels" 


Very_Long_E  0.074 

Long_E  0.138 

Medium_E  0.275 

Short_E  0.513 


"Priority  Vector  Of  Fuel  Usage  Levels'* 


High_F  0.143 

Medium_F  0.286 

Low  F  0.571 


"Priority  Vector  Of  Risk  Levels" 

High_R  0.143 

Medium_R  0.286 

Low  R  0.571 


"Priority  Vector  Of  Lethality  Levels" 

High_L  0.571 

Medium_L  0.286 

Low_L  0.143 


"Overall  Priority  Vector  Of  Criteria  Levels  With  Respect  To  Goal  Of  Choosing 

Most  Appropriate  Differential  Game" 


Very_High_S 

0.017 

High_S 

0.035 

Med_S 

0.069 

Low_S 

0.138 

One 

0.080 

Two 

0.020 

>  Two 

0.010 

Very_Long_E 

0.006 

LonaE 

0.012 

Medium  E 

0.023 
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Short  E 

0.043 

High_F 

0.011 

Medium_F 

0.023 

Low_F  “ 

0.045 

High_R 

0.049 

Medium_R 

0.098 

Low  R 

0.196 

High_L 

0.072 

Medium_L 

0.036 

Low_L 

0.018 

An  advantage  of  the  AHP  implementation  methodology  for  the  system 
identifier  is  that  incremental  development  is  easily  supported.  New  games  and 
parameterizations  can  easily  be  incorporated  by  simply  determining  the 
appropriate  level  under  each  criterion,  and  including  the  new  games  to  the 
scoring.  An  explanation  of  the  decision  is  also  available  by  examining  the 
different  layer  weights  with  respect  to  the  goal  of  choosing  the  best  differential 
game.  Risk  (0.343)  and  Dynamic  Sensitivity  (0.259)  are  the  most  important 
criteria  for  the  example.  Diff  Game  1  Param  2  (0.434)  had  the  best  level  value 
for  Dynamic  Sensitivity,  and  was  as  good  as  all  other  choices  for  Risk. 

One  area  of  difficulty  that  still  requires  research  for  the  AHP  based  method 
is  how  to  determine  levels  from  the  differential  game  and  parameterization  when 
the  criteria  are  not  part  of  the  game  formulation.  The  appropriate  criteria,  sub¬ 
criteria,  and  proper  number  of  levels  for  each  criterion  also  require  further 
investigation. 
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5.3  CONTROL  ADAPTER 

The  control  adapter  is  responsible  for  following  the  differential  game 
trajectories  from  the  goal  adapter,  and  for  firing  missiles  (reproduction).  In  our 
current  implementation  the  goal  adapter  is  a  rule  based  expert  system.  The  goal 
adapter  determines  the  aircraft  controls  that  best  follow  the  differential  game 
trajectory.  Techniques  for  following  a  known  trajectory  are  not  new  and  simply 
require  tradeoffs  when  a  trajectory  requires  controls  outside  the  aircraft  flight 
envelope. 

The  phase  1  simulator  currently  has  implemented  an  expert  system 
trajectory  follower.  It  determines  how  to  use  excess  capabilities  when  available. 
Suppose  that  we  use  a  two  dimensional  differential  game  (FI  6  pursuit  of  Harrier), 
the  controller  may  find  that  the  turn  called  for  is  wide  enough  that  excess  thrust 
and  lift  are  available.  The  controller  will  make  the  simpler  turn  called  for  by  the 
two-dimensional  game  and  use  the  excess  flight  capabilities  to  climb  or  dive  to 
bring  the  ownship  and  adversary  into  the  same  horizontal  plane  to  better  fit  the 
differential  game  model.  These  ideas  are  implemented  currently  in  the  control 
adapter  with  specific  rules  dependent  upon  the  differential  game  model  chosen. 
The  control  adapter  also  determines  if  a  missile  launch  opportunity  has  occurred 
by  reaching  the  terminal  manifold  or  the  differential  game,  or  a  firing  surface  of 
the  probabilistic  model  has  been  reached. 
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6.  DIFFERENTIAL  GAME  KNOWLEDGE  BASE 

This  section  will  cover  the  incorporation  of  differential  games  into  the 
knowledge  base.  Currently  our  phase  I  simulator  has  strategies  of  a  Harrier 
chasing  an  FI  6  (highly  maneuverable  but  slower  pursuer),  an  FI  6  chasing  a 
Harrier  (faster  pursuer  against  highly  maneuverable  evader),  and  line  of  sight 
strategies  (LOS).  The  LOS  strategies  consist  of  turning  at  the  maximum  rate 
available  to  the  aircraft  while  maintaining  a  minimum  desired  velocity.  There  are 
many  implementations  of  this  strategy  currently  available  such  as  proportional 
navigation  (PRONAV),  and  LOS  will  not  be  extensively  discussed  here.  The 
other  four  strategies  (a  pursuit,  and  an  evasion  strategy  for  each  game)  are 
extensively  discussed  and  the  game  of  degree  solutions  are  new  for  eccentric 
terminal  manifolds. 

Two  obstacles  prevent  the  building  of  a  large  knowledge  base  of 
differential  game  solutions.  The  first  one  deals  with  dimensionality  and 
discontinuity  of  trajectories  and  controls.  This  obstacle  has  prevented  the 
generation  of  solutions  to  all  but  very  simple  low  dimensional  problems.  The 
second  obstacle  is  the  need  for  feedback  control  laws  that  can  be  generated  in 
real  time.  The  normal  procedure  is  to  solve  a  corresponding  two  point  boundary 
value  problem  (TPBVP)  numerically.  This  numeric  solution  cannot  be  found  in 
real  time  and  is  the  open  loop  control.  Open  loop  control  does  not  allow  one  to 
take  advantage  of  an  opponent's  mistakes. 

Currently  research  in  McDonnel  Douglas  Missile  Component  Company's 
guidance  navigation  and  control  technology  group  is  exploring  the  use  of  Neural 
Nets  to  overcome  the  obstacles  to  the  use  of  differential  game  technology.  A 
brief  overview  of  Neural  Nets  is  found  in  section  5.2.2. 
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The  approach  being  taken  is  to  train  individual  Neural  Nets  to  be  the 
optimal  feedback  controllers  for  the  pursuer  and  evader.  The  only  information 
assumed  to  be  available  is  the  initial  state  of  the  system,  and  the  final  state  of  the 
system  via  simulation.  No  knowledge  of  the  true  optimal  trajectories  or  feedback 
control  strategies  is  assumed.  This  approach  allows  training  of  optimal  feedback 
control  strategies  without  explicit  knowledge  of  the  true  solution. 

FIGURE  6.1  shows  the  approach  to  training  the  neural  controllers.  Initial 
conditions  in  state  space  are  generated  randomly.  A  discretized  version  of  the 
system  is  used  along  with  the  current  parameterization  (weight  structure)  of  the 
neural  controllers  to  close  the  loop  via  simulation.  The  states  of  both  the 
controllers  and  system  are  generated  at  each  time  step  and  stored.  At  the  end  of 
the  simulation  the  terminal  cost  is  known. 

_ FIGURE  6.1 _ 

NEURAL  NET  TRAJECTORY  SHAPING 
VIA  DIFFERENTIAL  GAMES 


FORWARD  ACTIVATION  THROUGH  SIMULATION 


u. 


Neural 

SYSTEM 

Controier 

— 

•  •  • 


^<X,>  fo(X„>  fo(Xa) 

D»0  ^  K,  ^ 


BACKPROPAGATION  THROUGH  GRADIENT  DESCENT 
ACCUMULATION  OF  NEURAL  WEIGHT  ADJUSTMENTS 
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The  partials  of  this  cost  are  fed  back  thru  the  system  as  shown  in  the 
Figure  6.1 .  A  modified  version  of  the  backpropogation  algorithm  is  used  to  adjust 
the  neural  controller  weights.  The  backpropogation  algorithm  is  fed  for  each  time 
instance  of  the  discretized  system,  the  partial  derivative  of  the  terminal  cost  with 
respect  to  the  state  of  the  controller  at  that  time  instance.  The  backprogation 
algorithm  then  determines  parameter  adjustments  (weight)  for  that  time  instance, 
along  with  the  partial  of  the  terminal  cost  with  respect  to  the  controllers  input. 
The  modification  to  the  backpropogation  algorithm  comes  from  the  fact  the 
weight  adjustments  are  accumulated  over  all  time  steps  of  the  simulation.  After 
feeding  back  the  partial  for  all  time  steps  the  weights  of  the  neural  controllers  are 
adjusted. 

The  simulation  for  the  same  initial  conditions  is  repeated  with  the  modified 
neural  controllers.  The  adjustment  simulation  cycle  is  repeated  until  a  change  in 
the  terminal  cost  falls  below  a  predetermined  threshold.  This  cycle  is  then 
repeated  for  new  randomly  generated  initial  states.  The  learning  process  is 
terminated  when  only  minor  adjustments  are  necessary  for  newly  generated 
random  initial  conditions. 
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6.1  HARRIER  CHASING  AN  FI 6  (ECCENTRIC  AGGRESSIVE  PEDESTRIAN) 

The  first  differential  game  considered  is  that  of  an  infinitely  maneuverable 
pursuer  against  a  faster  evader  with  limited  maneuverability.  Asymptotically  this 
can  be  viewed  as  a  Harrier  (infinitely  maneuverable)  against  an  FI  6  (faster).  The 
game  takes  place  in  two  dimensional  planar  space.  This  game  can  be  used 
assuming  altitude  remains  nearly  constant  during  an  encounter.  Only  the  Harrier 
type  aircraft  is  assumed  to  have  aggressive  capabilities,  modeled  by  an  eccentric 
circle  terminal  manifold.  The  game  cf  degree  for  this  type  of  terminal  manifold  is 
solved  for  the  first  time. 


6.3.4.  MODEL 

The  encounter  takes  place  in  two-dimensional  realistic  space  (a  fixea 
inertial  coordinate  system).  The  state  space  dimension  is  five.  The  model  for  the 
infinitely  maneuverable  aircraft  (Harrier)  is: 


Xo*  5=  wpsin<!> 

(6.3.2) 

Y  p*  =  wpeosO 

where 

w2  *  speed  of  Harrier  type  aircraft 

<I>  s  control  heading  for  Harrier  type  aircraft, 
measured  from  y-axis 


The  model  for  the  faster  but  limited  maneuverable  aircraft  (FI  6)  is: 

X-j  *  =  w-|Sin0 
(6.3.3)  Y1*  =  w1cos0 

0*  =  w-|  <t»/R 

where 


w-j  3  speed  of  FI 6  type  aircraft 
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0  =  heading  of  FI  6  type  aircraft  measured  from  y-axis 
R  =  minimal  turn  radius  of  FI  6  type  aircraft 
0  s  control  for  FI  6  type  aircraft  (-1  <  0  <  1 ) 
note:  -1  (1 )  _  maximum  left  (right)  turn 

A  legitimate  game  requires  w-j  >  W2-  There  would  be  no  contest  if  the  Harrier 
type  aircraft  was  both  infinitely  maneuverable  and  faster.  The  two-dimensional 
realistic  space  model  is  shown  in  Figure  6.2. 

The  control  strategies  and  target  set  are  determined  in  a  reduced  state 
space.  The  dimension  of  the  reduced  state  space  is  two.  The  new  geometric 
coordinate  system  (a  relative  inertial  coordinate  system)  is  still  two-dimensional. 
A  right  hand  coordinate  system  with  the  new  y-axis  aligned  with  the  FI  6  type 
aircraft  velocity  vector  is  used.  The  new  coordinate  system  is  relative  to  the  FI  6 
type  aircraft. 


FIGURE  6.2 


Realistic  Space 
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State  equations  for  the  new  coordinate  system  are: 


(6.3.3) 


X*  - - Y4>  +  W£  sin(O-Q) 


Y*  =  -  X0  -  Wi  +  Wo  cosfCMi). 

R 


Without  loss  of  generality  we  introduce  the  new  control  for  the  Harrier,  <I>  =  O  - 
0.  This  leads  to: 


(6.3.4) 


X*  - - Yp  +  w2  sin(C>*) 


Y*  =  -  Xf  -  w1  f  w2  cos(C>*). 

R 


One  last  transformation  is  made  to  follow  Davidovitz  and  Shinar  (1985).  A 
change  of  coordinates  is  introduced  to  normalize  the  system  equations.  The  new 
coordinate  transformation  is: 


(6.3.5) 


X(t)  =  X(tR/w1)/R 
Y(t)  =  Y(tR/w-|)/R. 


This  leads  to  the  following  normalized  state  equations: 
X*  =  -  Y<j>  +  vsinO* 

(6.3.6) 

Y*  =  X<t>  - 1  +  vcosO 


where: 


/  =  the  speed  ratio  w2/w-| . 
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The  normalized  coordinate  system  relative  to  the  FI  6  type  aircraft  is  shown  in 
Figure  6.3. 


FIGURE  6.3 


COORDINATE  SYSTEM  RELATIVE  TO  F16  TYPE  AIRCRAFT 


6.3.5  STRATEGY 

Only  the  Harrier  type  aircraft  has  aggressive  capabilities.  The  terminal 
manifold  is  an  eccentric  circle  centered  at  the  FI  6  in  the  reduced  state  space  of 
equations  (6.3.6).  The  eccentricity  is  positive  along  the  Y-axis.  The  model  for 
the  encounter  is  shown  in  Figure  6.4.  The  goal  of  the  Harrier  type  aircraft  is  to 
penetrate  the  eccentric  "kill"  circle  centered  at  the  FI  6  type  aircraft. 
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FIGURE  6.4 


The  model  variables  are: 

b  s  Eccentricity  of  Harrier  type  kill  circle 
(3  =  Radius  of  Harrier  type  kill  circle 

v  =  Velocity  ratio:  v  <  1  (Harrier  type  aircraft  slower  than  FI  6  type  aircraft) 

0  3  Parameter  of  Harrier  type  kill  circle  (terminal  manifold) 

X,Y  s  Cartesian  coordinates  in  frame  of  the  FI  6  type  aircraft  with  F16  type 
velocity  aligned  with  y-axis 

The  reduced  state  equations  6.3.7  are  repeated  here  for  clarity: 

X*  =  -Yp  +  vsinO 

(6.3.8) 

Y*  =  Xp  -  1  +  vcosO 

where: 

O  3  Harrier  type  aircraft  control 
p  3  FI  6  type  aircraft  control:  |  p  j  <  1 . 
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The  solution  must  be  found  in  retro-time.  We  introduce  the  retro  time  variable  t  as 
follows: 

x  =  tf  —  t 

where: 

t  =  forward  time 

tf  =  final  time  (when  kill  circle  is  penetrated) 


The  final  conditions  are: 

Xf  =  psin(0) 

Yf  =  (3cos(0)  +  b 


(6.3.8) 


Section  2.2.2  page  32  showed  that  the  terminal  manifold  is  partitioned  into  the 
usable  part  (UP),  non-usable  part  (NUP),  and  boundary  of  the  usable  part  (BUP). 
The  BUP  and  UP  regions  are: 


BUP  s 


(Xf.Yf) :  0U(3  =  cos 


-1 


v  +  b(1  +b2-v2)1/2 
(Ub5) 


(6.3.9) 


UP  ■((Xj.Yf):  |0|  <  0up) 


The  next  step  is  to  determine  the  capture  region  and  barriers  defined  in 
section  2.2  page  32  (the  game  of  kind).  Clearly  the  barriers  enclose  the  capture 
region  as  a  subset  of  the  game  space  because  the  Harrier  type  aircraft  is  slower 
than  the  FI  6  type  aircraft.  When  the  velocity  vectors  of  both  the  Harrier  and  F16 
type  aircraft  are  aligned,  it  is  impossible  for  the  Harrier  to  close  distance.  The 
barrier  equations  for  the  right  half  plane  (RHP)  (SHINAR  19851)  are: 
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(6.3.11) 


X(x)  =  -bsinx  +  cost  -  1  +(vx  +  p)sin(6up  -  x) 
Y(x)  =  bcosx  +  sinx  +  (v't  +  {5)cos(0up  -  x). 


Paths  must  be  determined  for  all  points  lying  in  the  capture  region  by  the 
game  of  degree  (section  2.1 ).  A  cost  function  is  assigned  so  that  all  the  possible 
paths  terminating  on  the  kill  circle  can  be  compared.  The  cost  function  used  by 
the  Harrier  is  the  time  to  the  kill  circle  (terminal  manifold).  This  cost  function  is: 


(6.3.12) 


r(x,Y) 


dt 


Isaacs'  main  equation  (2.1.32)  is: 


(6.3.13) 

min  max[<|)(-Yrx+XrY)+v<rx2+rY2)1  ^2cos(0-tan 
<X>  <j> 


rx 

rY 


rY  + 1]  =  o 


This  leads  to  the  optimal  controls  for  the  right  half  plane  (RHP): 


(6.3.14) 


O  =  it  +  tan'1 


rx 

rY 


<J>  =  sign{-YTx  +  XTy)  =  sign{S} 


The  terminal  conditions  are: 

Xf  =  psin(0) 

Yf  =  |3cos(0)  +  b 
(6.3.15)  T(0)  =  O 

S  =  -bTx 
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0  =  TxCosG  -  Tysine  (Tranversality) 


These  terminal  conditions  leads  to  the  following  terminal  controls  and  costates  for 
the  RHP  : 

G>  =  n  +  0 
4>=  -1 


*X  = 


sin0 

v  +  cos0  -  bsine 


rY  = 


COS0 

v  +  cos0-  bsine 


The  retro  state  and  costate  equations  are: 


(6.3.16) 


rx°  =  rY0 

rY°  =  -rX4> 

X°  =  Y<t>  -  vein® 

Y°  =  — X<J>  +1  -  vcosO 


Integration  of  these  retro  equations  with  the  terminal  conditions,  yield  for  the 
RHP: 

O  =  7C  +  9  -  x 
0  =  -1 

X(x)  =  -bsinx  +  cost  -  1  +(\a  +  p)sin(0-x) 

Y(x)  =  bcosx  +  sinx  +  (n  +  p)cos(0-x) 


(6.3.17) 
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The  optimal  controls  are  found  by  inverting  the  last  two  equations  of 
(6.3.16)  to  determine  0  and  t.  Unfortunately  the  equations  are  non-linear  and  the 
inversion  must  be  done  numerically. 

An  important  point  to  note  is  that  the  barrier  is  a  smooth  imbedding  in  the 
one  parameter  (0)  family  of  paths.  This  is  important  when  a  differential  equation 
is  approximated  by  a  difference  equation.  This  smoothness  property  means  that 
the  strategies  played  cannot  change  drastically  during  an  integration  update 
period  for  the  approximating  difference  equation.  This  will  not  be  true  when  the 
FI 6  type  aircraft  is  considered  the  aggressor. 

One  final  note  is  the  y-axis  serves  as  a  dispersal  surface.  For  any  point 
on  the  y-axis  there  are  two  optimal  paths,  one  to  the  right  and  one  to  the  left. 
These  paths  are  mirror  images  of  each  other,  due  to  the  symmetry  of  the 
problem.  The  Harrier  wishes  to  stay  on  the  y-axis,  but  the  optimal  move  for  the 
FI  6  is  to  force  the  trajectory  off  the  y-axis.  The  barrier  and  optimal  trajectories 
are  shown  in  the  figure  below. 
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FIGURE  6.5 

Barriers  and  Optimal  Trajectories  For  The  Capture  Region  Harrier  type  Aircraft 

The  Aggressor 
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6.2.  FI  6  CHASING  A  HARRIER  (ECCENTRIC  HOMICIDAL  CHAUFFEUR) 

The  second  differential  game  considered  is  that  of  a  faster  pursuer  against 
a  slower  but  infinitely  maneuverable  evader.  The  role  of  pursuer  and  evader 
have  been  exchanged  from  that  of  section  6.3  (Harrier  Chasing  An  FI  6).  The 
FI  6  type  aircraft  is  now  assumed  to  be  the  aggressor.  This  game  also  takes 
place  in  two  dimensional  planar  space.  Only  the  FI  6  type  aircraft  is  now 
assumed  to  have  aggressive  capabilities,  modeled  by  an  eccentric  circle  terminal 
manifold.  The  game  of  degree  for  this  type  of  terminal  manifold  is  solved  for  the 
first  time. 

The  fixed  inertial  model  is  the  same  as  in  section  6.3.4  page  135.  The 
FI 6  type  aircraft  now  has  the  aggressive  capabilities.  The  terminal  manifold  is 
an  eccentric  "kill"  circle  centered  at  the  FI  6  in  the  reduced  state  space  (shown  in 
Figure  6.3).  The  eccentricity  is  positive  along  the  Y-axis.  The  goal  of  the  FI  6 
type  aircraft  is  to  force  the  Harrier  type  aircraft  to  penetrate  the  terminal  manifold. 
The  relative  coordinate  frame  model  is  shown  in  Figure  6.6. 
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FIGURE  6.6 


The  model  variables  are: 

a  s  Eccentricity  of  FI  6  type  kill  circle  (terminal  manifold) 

a  h  Radius  of  FI  6  type  kill  circle  (terminal  manifold) 

v  3  Velocity  ratio:  v  <  1  (Harrier  type  aircraft  slower  than  FI  6  type  aircraft) 

0  3  Parameter  of  FI  6  type  kill  circle  (terminal  manifold) 

X,Y  s  Cartesian  coordinates  in  frame  of  the  FI  6  type  aircraft  with  FI  6  type 
velocity  aligned  with  y-axis 

The  state  equations  and  controls  are  the  same  as  in  the  case  that  the 
Harrier  was  the  aggressor  (equations  (6.3.8)).  The  solution  again  must  be  found 
in  retro  time  (t).  The  terminal  conditions  are: 


(6.3.17) 


Xf  =asin0 
Yf  =  acos0  +  a 
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The  usable  part  (UP)  and  the  boundary  of  the  usable  part  (BUP)  of  the  terminal 
manifold  (kill  circle)  are: 


(Xf , Yf) .  0yQ£  =  cos  ^ 

-v+  a(1+a2-v2)1/2 

BUP  s 

(1+a2) 

(6.3.18) 

UP  s  ((XflYf):  |0|  <0U(X) 


Two  initial  assumptions  will  be  used  here.  The  first  is  that  the  whole  space  is  the 
capture  region.  This  implies  that  the  barriers  (if  they  exist)  do  not  meet  (are 
open).  The  mathematical  condition  for  open  barriers  from  Shinar  (1985)  is: 

(6.3.1 9)  a  >  (1  +  a2  -  v2)1  /2  +  vsin*1  (cos0ua)  -  1 

The  second  condition  is  that  the  barriers  meet  the  "kill"  circle  tangentially.  This 
mathematical  condition  from  Shinar  (1985)  is: 


(6.3.20)  a2  >  (1  +  a2  -  v2) 
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The  barrier  equations  for  the  right  half  plane  (RHP)  from  Shinar  (1985)  are: 

X(x)  =  asint  -  cost  +  1  +  (a  -  vt)sin(x  +  0llfY), 

(6.3.21) 

Y(t)  =  acosx  +  sint  +  (a  -  vx)cos(t  +  0ua), 
where  x  <  2U  -  cos'1  (v)  -  0^ 

A  cost  function  is  again  needed  to  choose  a  trajectory  leading  to  the 
terminal  manifold.  The  cost  function  used  by  the  Harrier  is  the  time  to  the  kill 
circle  (terminal  manifold),  but  in  this  case  it  is  the  time  till  the  Harrier  type  aircraft 
reaches  the  FI  6  kill  circle.  The  cost  function  is: 


r(X,Y)  = 


Isaacs'  main  equation2  (2.1.32)  is  : 


(6.3.22) 


I  V 

max  min  [<})(-Yrx+XrY)+v(rx2+rY2)1/^cos(0-tan'1  -  J-Ty+I]  =  0 


O  <f> 


^The  barrier  termination  time. 

2The  role  of  minimizer  and  maximizer  have  now  been  switched  form  the  case 
where  the  Harrier  type  aircraft  is  the  aggressor. 
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This  leads  to  the  optimal  controls  for  the  right  half  plane  (RHP): 


<!>  =  tan'1 


rX 


(6.3.23) 


<}>  =  -sign{-YTx  +  XTy}  =  -sign{S} 


The  games  space  must  be  decomposed  according  to  section  2.1.6  page 
13.  Each  of  the  cases  discussed  below  corresponds  to  a  region  Ejj  in  which  the 
optimal  controls  are  continuous.  The  singular  surfaces  that  function  as  borders 
(controls  are  discontinuous)  for  the  different  cases  correspond  to  the  /U|j. 
Equation  (2.1.47)  is  used  to  generate  the  initial  conditions  on  the  singular 
surfaces. 

The  first  case  to  be  considered  is  the  Harrier  type  aircraft  below  the 
barrier.  This  corresponds  to  the  classical  situation  in  which  the  pursuer  (FI  6) 
must  perform  a  swerve  maneuver.  A  universal  surface  exists  along  the  y-axis 
both  above  and  below  the  kill  circle  (terminal  manifold).  The  y-axis  below  the 
kill  circle  is  a  universal  surface  because  by  symmetry  there  are  trajectories 
entering  from  each  half  plane.  These  trajectories  upon  entry  continue  along  the 
universal  surface3.  This  corresponds  to  a  tail  chase  with  the  Harrier  type  aircraft 
chasing  the  FI  6  type. 


3A  further  discussion  of  universal  surfaces  and  their  implication  can  be  found  in 
Issaacs  (2). 
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The  universal  surface  below  the  y-axis  will  be  used  as  an  intermediate 
terminal  manifold  to  generate  the  swerve  maneuver.  The  terminal  conditions  are 
with  parameter  s,  (s  <  a  -  a  <  0): 

Xf  =0 
Yf  =  s 

(6.3.25) 

(s  -  (a  -  a)) 

T(s)  = -  +  constant  (tranversality) 

(1-v) 

rx 

max  min  [<j>(-srx)+v(rx2+rY2)1/2cos(0-tan'1  -  )-ry+1]  =  0 

O  4>  Ty 

These  terminal  conditions  lead  to  the  following  terminal  controls  and  costates  for 
the  RHP: 

rY  =  (i  -  v)-i 
rx  =  o 

(6.3.26) 

oF  =  0 
<f)F  =  -1 
Sp  =  0. 

The  retro4  state  and  costate  equations  are: 

rx°  =  rY  <j> 

rY°  *rX$ 

(6.3.27)  X°  =  Y0  -  vsinO 

Y°  =  — X<f>  +  1  -  vcosO 

s°  =-rx 

4Retro  time  x  is  now  the  time  to  the  universal  surface,  not  the  time  to  the  Kill 


circle. 
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Integration  of  these  retro  equations  with  the  terminal  conditions  yield  for  the 
RHP: 


(6.3.27) 


<D(x)  =  -x 
4>(x)  =  -1 

X(x)  =  (vt  -  s)sinx  +  cosx  -  1 
Y(t)  =  — (vx  -  s)cosx  +  sinx 


Solving  for  s  and  x^: 

s  =  ((X  +  I)2  +  Y2-  1  )1/2  +  tx 

(6-3.28)  i  i  o  p  MO 

x  =  tan'1  (Y/(X+1 ))  +  tan'1  (((X+1  )2  +  Y2  -  1  )1/2) 

It  should  also  be  noted  at  this  point  that  extension  of  the  barrier  equations 
from  the  barrier  termination  point  at  x  *  2n  -  cos'^v)  -  0ua  till  the  barrier 
extension  intersects  the  negative  Y-axis,  is  a  singular  surface.  Once  a  trajectory 
starting  below  the  barrier  reaches  this  surface  the  swerve  portion  of  the 
maneuver  can  be  started.  At  this  point  the  pursuer  (FI  6)  begins  to  turn  to  start 
chasing  the  evader  (Harrier),  while  the  Harrier  also  starts  turning  to  begin 
evading. 

There  are  two  cases  to  be  considered  for  the  swerve  portion  of  the 
maneuver,  trajectories  that  terminate  directly  on  the  terminal  manifold,  and 
trajectories  that  first  terminate  on  a  universal  surface  consisting  of  the  Y-axis 
above  the  terminal  manifold  and  then  continuing  on  the  Y-axis  to  the  terminal 
manifold. 


5The  dependency  of  x  only  on  the  states  X,Y  allows  the  use  of  state  feedback  for 
cD. 
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We  now  consider  the  case  where  trajectories  terminate  directly  on  the 
terminal  manifold.  The  main  equation  and  optimal  control  solution  equations  are: 


(6.3.30) 

max  min[4>(-Yrx+XrY)+Krx^+TY^)1^cos(0-tan'^ 
<1>  0 


rY 


)-rY+i]  =  o 


(6.3.31) 


O  =  tan"1 


rY 


d  =  -sign{-YTx+XrY}  =  -~sign{S} 


The  terminal  conditions  with  parameter  0  (0  <  0  <  0ua)  are: 

Xf  =asin(0) 

Yf  =  acos(0)  +  a 

(6.3.32)  T(0)  =  0 

rxcos(0)  -  TYSin(0)  =  0 
S  =  -aTx 

These  terminal  conditions  lead  to  the  following  terminal  controls  and  costates  for 
the  RHP: 

TX  =  sin(0) 

Ty  =  cos(0) 

(6.3.33) 

0  =  0 
4>  =  1 


The  retro  state  and  costate  equations  are: 

rX°  =  *rY 
rY°  =  rX 

(6.3.34)  X°  =  -Y  +  vsmx 


Y°  =  X  +  1  -  VC0ST 
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S°  =  -rx 

Integration  of  these  retro  equations  with  the  terminal  conditions,  yield  for  the 
RHP: 

Tx  =  sin(t  +  8) 

Ty  =  cos(x  +  8) 

S  =  -cos(x  +  8)  -  a  +  cos(8) 

(6.3.35) 

O  =  8  +  x 
<(,=  1 

X(x)  =  (a  -  vx)sin(8+x)  +  asinx  -  cosx  +  1 
Y(x)  *  (a  -  voc)cos(8+x)  +  acosx  +  sinx 

Once  again  to  determine  the  optimal  controls  we  must  determine  x  and  8  from  the 
equations  for  X(x)  and  Y(x)  above.  The  problem  is  that  a  closed  form  solution 
does  not  exist  and  numerical  methods  must  be  relied  on. 

The  last  case  to  be  considered  is  for  those  trajectories  that  terminate  on 
the  Y-axis  universal  surface  above  the  terminal  manifold.  The  encounter  then 
ends  with  a  classical  tail  chase  of  the  pursuer  (FI  6)  chasing  the  evader  (Harrier). 
The  main  equation  and  optimal  control  solution  equations  remain  the  same  as  in 
the  case  of  termination  directly  on  the  terminal  manifold.  The  difference  is  that 
the  Y-axis  above  the  terminal  manifold  now  serves  as  an  intermediate  terminal 
manifold  for  the  trajectories.  This  means  that  we  are  now  looking  for  trajectories 
that  terminate  on  the  following  surface  parameterized  by  o  (a  +  a  <  a): 

Xf(a)  =  0 

(6.3.36) 

Yf(a)  =  a 


The  other  terminal  condition  is: 
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integration  of  these  retro  equations  wiih  the  terminal  conditions,  yield  for  the 
RHP: 

Tx  =  sin(x  +  8) 

Ty  =  cos(x  +  0) 

S  =  -cos(x  +  0)  -  a  +  cos(0) 

(6.3.34) 

O  =  0  +  x 
0=  1 

X(x)  =  (a  -  vx)sin(0+x)  +  asinx  -  cosx  +  1 
Y(x)  =  (a  -  \n) cos(0+x)  +  acosx  +  sinx 

Once  again  to  determine  the  optimal  controls  we  must  determine  x  and  8  from  the 
equations  for  X(x)  and  Y(x)  above.  The  problem  is  that  a  closed  form  solution 
does  not  exist  and  numerical  methods  must  be  relied  on. 

The  last  case  to  be  considered  is  for  those  trajectories  that  terminate  on 
the  Y-axis  universal  surface  above  the  terminal  manifold.  The  encounter  then 
ends  with  a  classical  tail  chase  of  the  pursuer  (FI  6)  chasing  the  evader  (Harrier). 
The  main  equation  and  optimal  control  solution  equations  remain  the  same  as  in 
the  case  of  termination  directly  on  the  terminal  manifold.  The  difference  is  that 
the  Y-axis  above  the  terminal  manifold  now  serves  as  an  intermediate  terminal 
manifold  for  the  trajectories.  This  means  that  we  are  now  looking  for  trajectories 
that  terminate  on  the  following  surface  parameterized  by  o  (a  +  a  <  o): 

Xf(o)  =  0 

(6.3.35) 

Yf(o)  =  a 


The  other  terminal  condition  is: 
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(6.3.37)  r{o)  =  (o  -  (a+a))/(1  -  v) 

This  leads  to  the  following  terminal  controls  and  costates  fcr  the  PHP: 
rY  =  1/(1  -  v) 

0  =  0 

(6.3.38) 

0  =  1 

Tx  =  0  (continuity  on  US) 

The  retro  state  and  costate  equations  are: 

rx°  =  rY<t> 
rY°  =  -rX0 

(6.3.39)  S°  =  -rx 

X°  =  -Y  +  vsinx 
Y°  =  X  +  1  -  VCOST 

Integration  of  these  retro  equations  with  the  terminal  conditions,  yield  for  the 
RHP: 

sin(x) 

rX=  - — 

1  -  v 

cos(x) 
rY=  - — 

1  -  V 

S  =  (cos(x)  -  1)/(1  -  v) 

(6.3.40)  O  =  x 


0=  1 

X(x)  =  csinx  -  cost  -  vxsinx 
Y(x)  =  ocosx  +  sinx  -  vxcosx 
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The  closed  loop  optimal  control  can  be  found  directly  from  the  last  two  equations 
as: 


(6.3.41)  x  =  tan*1  ((X-1  )/Y)  +  tan*1  (((X-1  )2  +  Y2  -  1  )~1/2) 


The  barrier  and  optimal  trajectories  are  shown  in  the  Figure  6.7. 


FIGURE  6.7 

Barriers  and  Optimal  Trajectories  For  The  Capture  Region 
F16  type  Aircraft  The  Aggressor 
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7.  CONTRIBUTIONS,  SUMMARY  AND  FUTURE  RESEARCH 

We  have  extensively  explored  the  use  of  low  order  differential  games  in 
flight  and  fire  control  via  artificial  intelligence  methodologies.  Differential  game 
technology  currently,  and  for  the  foreseeable  future,  will  only  be  able  to  generate 
solutions  to  low  dimensional  models.  Unfortunately,  these  low  dimensional 
models  by  themselves  are  not  usable  for  actual  air-combat.  This  dissertation  has 
developed  methodologies  for  dealing  with  extremely  complex  dynamical  systems 
(such  as  air-combat)  with  only  a  partial  understanding  of  the  true  system  nature. 

We  have  given  an  appropriate  derivation  of  the  necessary  conditions  for  the 
differential  game  of  kind  and  game  of  degree.  Concentration  has  been  on 
differential  game  theory  actually  applicable  to  air-combat.  A  simplified 
development  of  the  game  of  degree  necessary  conditions  combines  Berkovitz’s 
methodology  with  Isaacs'  theorem  {that  all  games  are  equivalent  to  an 
autonomous  game  with  terminal  payoff).  The  consideration  of  only  autonomous 
feedback  strategies  when  dealing  with  autonomous  systems  has  extended 
Berkovitz's  results,  which  depended  always  on  using  the  family  of  time  varying 
feedback  strategies  in  the  variation. 

We  have  also  explored  the  nature  of  complex  systems  in  which  the  form 
and  order  changes.  This  exploration  has  reviewed  and  extended  Mesarovic’s 
definition  of  a  general  system.  Traditional  system  theory  falls  under  Mesarovic's 
definition  in  which  the  relations  and  components  remain  fixed.  Complex  systems 
such  as  air-combat  do  not  fall  in  this  category.  The  addition  (reproduction)  and 
subtraction  (attrition)  of  components  modifies  the  system  relations.  Because  of 
this  we  have  extended  Mesarovic's  definition  to  handle  self  modifying  systems 
and  have  termed  these  systems  Semantic  Systems. 
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We  have  shown  that  Semantic  Systems  naturally  relate  to  the  Semantic 
Control  Paradigm.  Future  research  is  still  open  concerning  conditions  under 
which  key  Semantic  System  Components  are  survivable  and  reliable. 
Survivability  relates  to  the  conditions  under  which  a  component  can  prevent  its 
own  removal  from  the  system  (semantic  order).  Reliability  relates  to  the 
conditions  under  vfrhich  a  component(s)  can  force  a  particular  semantic  state. 
Survivability  and  reliability  are  the  Semantic  System  extensions  of  reachability 
and  controllability. 

In  addition,  we  have  also  created  a  knowledge  base  of  differential. 
Derived  for  the  first  time  were  the  game  of  degree  solutions  of  an  eccentric 
terminal  manifold  for  a  Harrier  evader  and  FI  6  pursuer.  These  game  solutions 
are  currently  in  our  air-combat  simulator  derived  under  our  new  Frame  Based 
Simulation  methodology  for  Semantic  Systems.  Current  involvement  includes 
research  to  increase  the  knowledge  base  of  game  solutions  via  artificial  neural 
nets.  The  approach  is  to  train  individual  Neural  Nets  to  be  the  optimal  feedback 
controllers  for  the  pursuer  and  evader.  The  only  information  assumed  to  be 
available  is  the  initial  state  of  the  system,  and  the  final  state  of  the  system  via 
simulation.  This  approach  assumes  no  knowledge  of  the  true  optimal  trajectories 
or  feedback  control  strategies.  This  approach  allows  training  of  optimal  feedback 
control  strategies  without  explicit  knowledge  of  the  true  solution. 

We  have  explored  artificial  intelligence  methods  to  determine  the  most 
appropriate  differential  game  from  the  knowledge  base.  Development  has  also 
included  a  probabilistic  missile  effectiveness  model,  and  a  methodology  to  get 
around  the  zero-one  terminal  manifold  inherent  in  differential  games.  The  zero- 
one  terminal  manifold  means  a  missile  has  zero  probability  of  destroying  an 
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adversary  until  it  reaches  a  surface  at  which  point  the  missile  has  100% 
probability  of  destroying  the  adversary. 

The  general  approach  taken  in  this  work  has  been  to  combine  different 
technologies  that  by  themselves  are  incapable  of  solving  the  problem  at  hand. 
Hybridization  techniques  for  future  integrated  system  methodologies  will  solve 
problems  that  are  currently  beyond  the  scope  of  pure  solutions.  Methodologies 
in  this  work  have  application  to  problems  currently  deemed  so  difficult  that  no 
appropriate  mathematical  model  can  be  derived,  nor  can  be  found  any 
appropriate  classical  analytical  methods. 

We  have  outlined  three  Al  methodologies  for  splicing  together  the 
differential  games;  rule  base  expert  systems,  artificial  neural  nets,  and  the 
Analytical  Hierarchy  Process  (AHP).  The  rule  based  expert  system  appears  to 
be  the  easiest  methodology  for  immediate  implementation,  but  requires  a  deep 
understanding  of  the  game  in  the  knowledge  base  and  is  brittle  to  incrementing 
the  knowledge  base  with  more  games.  The  artificial  neural  net  implementation 
offers  very  high  performance  via  training,  but  training  sets  would  be  very  difficult 
go  generate.  The  AHP  appears  to  offer  the  best  compromise.  It  requires  only 
local  understanding  via  the  pairwise  comparison  and  its  hierarchic  structure.  The 
AHP  also  would  easily  incorporate  incrementing  the  knowledge  base.  The 
question  of  appropriate  criteria  for  the  hierarchies  of  the  AHP  are  still  open  to 
research.  Training  methodologies  for  the  artificial  neural  net  approach  also  are 
still  open.  Lastly  a  comparison  via  simulation  of  the  performance  of  the  three 
methodologies  is  still  forthcoming. 
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10.  APPENDIX  A:  SIMULATOR  RESULTS 


This  section  contains  two  example  runs  from  the  simulator.  The  first 
example  shows  the  result  of  choosing  the  wrong  strategy.  The  pursuer  chooses 
a  line  of  sight  strategy  and  tries  to  turn  too  quickly  and  is  unable  to  capture  the 
evader.  The  second  example  shows  the  result  of  a  proper  choice  of  strategies  by 
both  players. 
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